Apparatus and method for processing information signal, apparatus and method for producing coefficient, apparatus and method for producing lookup table, program for performing each method, and medium recording each program

ABSTRACT

This invention relates to an apparatus for processing an information signal etc. that, when converting, for example, SD signal into HD signal, enables well to be obtained pixel data of HD signal no matter whether the dynamic range DR is large or small. DR in a class tap is detected. If DR≧Th, items of pixel data y 1-a −y 4-a  calculated by using item of coefficient data W i-a  corresponding to a class code Ca are estimated as items of pixel data of HD signal. If DR&lt;Th, an addition mean value of items of pixel data y 1-a −y 4-a , y 1-b −y 4-b  calculated by using items of coefficient data W i-a , W i-b  corresponding to class codes Ca, Cb is estimated as item of the pixel data of HD signal. The items of coefficient data Wi-a, Wi-b are obtained by learning between a student signal corresponding to the SD signal and a teacher signal corresponding to the HD signal by using a portion of the DR having a value thereof that is not less than the threshold value Th. The code Ca is converted into the code Cb so that the addition mean value can most approach a true value of the pixel data of the HD signal.

TECHNICAL FIELD

The present invention relates to, for example, an apparatus and a methodfor processing an information signal, an apparatus and method forproducing a coefficient, an apparatus and method for producing a lookuptable, program for performing each method, and a medium that recordseach program, which can well be applied to conversion, into a signal (HDsignal) corresponding to a high resolution, of, for example, a standardTV signal (SD signal) corresponding to a standard resolution or a lowresolution.

More specifically, it relates to the apparatus for processing theinformation signal etc. for, when converting a first information signalcomprised of multiple items of information data into a secondinformation signal comprised of multiple items of information data,extracting as a class tap multiple items of information data located ina periphery of a target position in the first information signal and, ifa dynamic range obtained from the information data in this class tapbelongs to one range, obtaining information data that constitutes thesecond information signal by using coefficient data that corresponds toa first class code obtained by class-categorizing this class tap basedon a result of learning between a student signal (a first learningsignal) and a teacher signal (a second learning signal) by use of such aportion of the dynamic range as to belong to that one range and, if thedynamic range belongs to another range different from that one range,obtaining the information data that constitutes the second informationsignal by performing addition mean on the information data calculated byusing coefficient data that corresponds to the first class code andinformation data calculated by using coefficient data that correspondsto a second class code obtained by converting this first class code,thereby enabling well to be obtained the information data thatconstitutes the second information signal no matter whether the dynamicrange is large or small.

The present invention relates also to the apparatus for processing theinformation signal etc. for, when converting a first information signalcomprised of multiple items of information data into a secondinformation signal comprised of multiple items of information data,extracting as a class tap multiple items of information data located ina periphery of a target position in the first information signal and, ifa dynamic range obtained from the information data in this class tapbelongs to one range, obtaining information data that constitutes thesecond information signal by using coefficient data that corresponds toa first class code obtained by class-categorizing that class tap basedon a result of learning by use of such a portion of the dynamic range asto belong to that one range between a student signal (a first learningsignal) and a teacher signal (a second learning signal) and, if thedynamic range belongs to another range different from that one range,obtaining the information data that constitutes the second informationsignal by using coefficient data based on a result of learning betweenthe student signal and the teacher signal without performing anyclass-categorization thereof, thereby enabling well to be obtained theinformation data that constitutes the second information signal.

BACKGROUND ART

Recently, a variety of proposals have been made for a technology toimprove a resolution, a sampling frequency or the like of an imagesignal or an audio signal. For example, in the case of up-converting astandard TV signal corresponding to a standard resolution or a lowresolution into a so-called HDTV signal with a high resolution or thecase of performing sub-sampling interpolation, it is known that betterperformance-wise results can be obtained by performing conversionprocessing involving class categorization rather than performing aconventional method by means of linear interpolation (see JapanesePatent Application Publication No. Hei 7-95591 and Japanese PatentApplication Publication No. 2000-59740).

This conversion processing involving class categorization relates to atechnology such that, for example, when converting a standard TV signal(SD signal) corresponding to a standard resolution or a low resolutioninto a signal (HD signal) corresponding to a high resolution, a class towhich pixel data at a target position in the SD signal belongs isdetected and, by using coefficient data that corresponds to this class,pixel data of the HD signal corresponding to the target position in theSD signal is produced from multiple items of pixel data of the SD signalbased on an estimate equation. The coefficient data used in thisconversion processing involving class categorization is determined bylearning such as the least-square method for each of the classesbeforehand.

It is to be noted that the coefficient data used in that conversionprocessing involving class categorization has been based on a result oflearning without performing any categorization in accordance withwhether a dynamic range is large or small, which dynamic range is adifference between a maximum value and a minimum value of multiple itemsof pixel data that constitute a class tap.

In this case, in view of a structure of the least-square method, suchcoefficient data is created as to reduce an error at a portion with ahigher frequency, that is, a portion with a smaller dynamic range.Therefore, at a portion with a larger frequency, that is, a portion witha lager dynamic range, an error with respect to a true value of thepixel data of the HD signal calculated by the estimate equation isliable to be smaller.

DISCLOSURE OF THE INVENTION

It is an object of the present invention to well obtain information datathat constitutes a second information signal no matter whether a dynamicrange is large or small.

An apparatus for processing an information signal according to theinvention is an apparatus for processing an information signal thatconverts a first information signal comprised of multiple items ofinformation data into a second information signal comprised of multipleitems of information data, the apparatus comprising class tap extractionmeans for extracting as a class tap multiple items of information datalocated in a periphery of a target position in the first informationsignal based on the first information signal, class categorization meansfor obtaining a first class code by categorizing the class tap extractedby the class tap extraction means as any one of a plurality of classesbased on the class tap, dynamic range processing means for detecting adynamic range which is a difference between a maximum value and aminimum value of the multiple items of information data contained in theclass tap extracted by the class tap extraction means based on the classtap, to obtain area information that indicates which one of a pluralityof sub-divided areas obtained by dividing a possible area of the dynamicrange into plural ones the dynamic range belongs to, class codeconversion means for converting the first class code obtained by theclass categorization means into one or a plurality of second class codeseach corresponding to the first class code, prediction tap extractionmeans for extracting as a prediction tap multiple items of informationdata located in a periphery of the target position in the firstinformation signal based on the first information signal, firstcoefficient data generation means for generating first coefficient data,which is used in an estimate equation corresponding to the first classcode obtained by the class categorization means, second coefficient datageneration means for generating second coefficient data, which is usedin the estimate equation, corresponding to one or the plurality ofsecond class codes, respectively, obtained through conversion by theclass code conversion means, first computation means for calculatinginformation data based on the estimate equation, by using the firstcoefficient data generated by the first coefficient data generationmeans and the prediction tap extracted by the prediction tap extractionmeans, second computation means for calculating information data basedon the estimate equation, by using the second coefficient data generatedby the second coefficient data generation means and the prediction tapextracted by the prediction tap extraction means, and addition means foroutputting the information data calculated by the first computationmeans as information data that constitutes the second information signalcorresponding to a target position in the first information signal ifthe dynamic range belongs to one sub-divided area according to the areainformation obtained by the dynamic range processing means and, if thedynamic range belongs to another sub-divided area different from the onesub-divided area, outputting data obtained by performing addition meanon the information data calculated by the first computation means andthat calculated by the second computation means as the information datathat constitutes the second information signal corresponding to thetarget position in the first information signal, wherein the firstcoefficient data generated by the first coefficient data generationmeans and the second coefficient data generated by the secondcoefficient data generation means are based on a result of learningbetween a first learning signal that corresponds to the firstinformation signal and a second learning signal that corresponds to thesecond information signal by use of such a portion of the dynamic rangeas to belong to the one sub-divided area, and wherein the class codeconversion means converts the first class code into the second classcode in such a manner that the addition mean value of the informationdata calculated by the first computation means corresponding to thefirst class code and the information data calculated by the secondcomputation means corresponding to the second class code may mostapproach a true value of the information data that constitutes thesecond information signal.

A method for processing an information signal according to the inventionis a method for processing an information signal that converts a firstinformation signal comprised of multiple items of information data intoa second information signal comprised of multiple items of informationdata, the method comprising a class tap extraction step of extracting asa class tap multiple items of information data located in a periphery ofa target position in the first information signal based on the firstinformation signal, a class categorization step of obtaining a firstclass code by categorizing the class tap extracted by the class tapextraction step as any one of a plurality of classes based on the classtap, a dynamic range processing step of detecting a dynamic range whichis a difference between a maximum value and a minimum value of themultiple items of information data contained in the class tap extractedby the class tap extraction step based on the class tap, to obtain areainformation that indicates which one of a plurality of sub-divided areasobtained by dividing a possible area of the dynamic range into pluralones the dynamic range belongs to, a class code conversion step ofconverting a first class code obtained by the class categorization stepinto one or a plurality of second class codes each corresponding to thefirst class code, a prediction tap extraction step of extracting as aprediction tap multiple items of information data located in a peripheryof the target position in the first information signal based on thefirst information signal, a first coefficient data generation step ofgenerating first coefficient data, which is used in an estimate equationcorresponding to the first class code obtained by the classcategorization step, a second coefficient data generation step ofgenerating second coefficient data, which is used in the estimateequation, corresponding to one or the plurality of second class codes,respectively, obtained through conversion by the class code conversionstep, a first computation step of calculating information data based onthe estimate equation, by using the first coefficient data generated bythe first coefficient data generation step and the prediction tapextracted by the prediction tap extraction step, a second computationstep of calculating information data based on the estimate equation, byusing the second coefficient data generated by the second coefficientdata generation step and the prediction tap extracted by the predictiontap extraction step, and an addition step of outputting the informationdata calculated by the first computation step as information data thatconstitutes the second information signal corresponding to a targetposition in the first information signal if the dynamic range belongs toone sub-divided area according to the area information obtained by thedynamic range processing step and, if the dynamic range belongs toanother sub-divided area different from the one sub-divided area,outputting data obtained by performing addition mean on the informationdata calculated by the first computation step and that calculated by thesecond computation step as the information data that constitutes thesecond information signal corresponding to the target position in thefirst information signal.

The first coefficient data generated by the first coefficient datageneration step and the second coefficient data generated by the secondcoefficient data generation step are then based on a result of learningbetween a first learning signal that corresponds to the firstinformation signal and a second learning signal that corresponds to thesecond information signal by use of such a portion of the dynamic rangeas to belong to the one sub-divided area, and in the class codeconversion step, the first class code is converted into the second classcode in such a manner that the addition mean value of the informationdata calculated by the first computation step corresponding to the firstclass code and the information data calculated by the second computationstep corresponding to the second class code may most approach a truevalue of the information data that constitutes the second informationsignal.

Further, a program related to the present invention causes a computer toperform the above-described method for processing the informationsignal. A computer-readable medium related to the present inventionrecords this program.

In the present invention, a first information signal comprised ofmultiple items of information data is converted into a secondinformation signal comprised of multiple items of information data. Itis to be noted that an information signal refers to, for example, animage signal comprised of multiple items of pixel data (sample data), anaudio signal comprised of multiple items of sample data, etc.

In this case, based on the first information signal, multiple items ofinformation data located in a periphery of the target position in thisfirst information signal are extracted as a class tap. Then, this classtap is categorized as any one of a plurality of classes, to obtain afirst class code.

Further, a dynamic range is detected which is a difference between amaximum value and a minimum value of the multiple items of informationdata contained in this class tap, to obtain area information thatindicates which one of a plurality of sub-divided ranges into which apossible range of the dynamic range is divided into plural ones thisdynamic range belongs to. For example, the area information indicatingwhether a dynamic range is less than or not less than a threshold valueis obtained.

The first class code obtained by categorizing the above-described classtap is converted into one or a plurality of second class codescorresponding to this first class code. For example, the conversion isperformed by referencing a lookup table in which a correspondencerelationship between the first class code and the second class code isstored.

In this case, the first class code is converted into the second classcode so that an addition mean value of information data calculated byusing first coefficient data that corresponds to the first class codeand information data calculated by using second coefficient data thatcorresponds to the second class code may most approach a true value ofthe information data that constitutes the second information signal.

Further, the first coefficient data is generated which is used in anestimate equation that corresponds to a first class code obtained bycategorizing the above-described class tap, while the second coefficientdata is generated which is used in the estimate equation thatcorresponds to one or a plurality of second class codes, respectively,obtained by converting the first class code as described above.

It is to be noted that the first coefficient data and the secondcoefficient data are based on a result of learning between a firstlearning signal that corresponds to the first information signal and asecond learning signal that corresponds to the second information signalby use of such a portion of a dynamic range as to belong to onesub-divided range.

For example, coefficient data of each class obtained beforehand, whichis used in the estimate equation, is stored in storage means, from whichcoefficient data that corresponds to a class indicated by a class codeis read.

Further, for example, coefficient seed data of each class obtainedbeforehand, which is coefficient data in a production equation, whichincludes a predetermined parameter, for producing coefficient data usedin the estimate equation is stored in the storage means, so thatcoefficient data, which is used in the estimate equation, can beproduced on the basis of the production equation by using thecoefficient seed data that corresponds to a class indicated by a classcode stored in this storage means.

Further, based on the first information signal, multiple items ofinformation data located in a periphery of a target position in thisfirst information signal are extracted as a prediction tap. Then, thefirst coefficient data produced as described above and this predictiontap are used to calculate information data based on the estimateequation. Furthermore, the second coefficient data produced as describedabove and this prediction tap are used to calculate information databased on the estimate equation.

If a dynamic range belongs to one sub-divided area, information datacalculated by using the first coefficient data as described above isoutput as information data that constitutes the second informationsignal corresponding to a target position in the first informationsignal. If the dynamic range belongs to another sub-divided rangedifferent from the one sub-divided range, data obtained by performingaddition mean on items of the information data calculated byrespectively using the first coefficient data and the second coefficientdata as described above is output as information data that constitutesthe second information signal corresponding to the target position inthe first information signal.

For example, if the dynamic range is not less than a threshold value,the information data calculated by using the first coefficient data isoutput and, if the dynamic range is less than a threshold value, dataobtained by performing addition mean on items of the information datacalculated by respectively using the first and second coefficient datais output.

In such a manner, in the present invention, if the dynamic range belongsto one sub-divided area, information data calculated by using the firstcoefficient data obtained so as to correspond to the first class codeobtained on the basis of a class tap is output as information data thatconstitutes the second information signal. In this case, the firstcoefficient data is based on a result of learning between a firstlearning signal that corresponds to the first information signal and asecond learning signal that corresponds to the second information signalby use of such a portion of the dynamic range as to belong to the onesub-divided range, to enable the information data that constitutes thesecond information signal to be accurately obtained.

Further, in the present invention, if the dynamic range belongs toanother sub-divided range different from the one sub-divided range, dataobtained by performing addition mean on items of information datacalculated by respectively using the first coefficient data obtained asto correspond to the first class code and the second coefficient dataobtained so as to correspond to the second class code obtained byconverting this first class code is output as information data thatconstitutes the second information signal. In this case, the first classcode is converted into the second class code so that an addition meanvalue of the information data calculated by using the first coefficientdata obtained so as to correspond to the first class code and theinformation data calculated by using the second coefficient dataobtained so as to correspond to the second class code may most approacha true value of the information data that constitutes theabove-described second information signal, thereby enabling accuratelyto be obtained the information data that constitutes the secondinformation signal.

It is thus, in the present invention, possible to well obtain theinformation data that constitutes the second information signal nomatter whether the dynamic range is large or small.

An apparatus for processing an information signal according to theinvention is an apparatus for processing an information signal thatconverts a first information signal comprised of multiple items ofinformation data into a second information signal comprised of multipleitems of information data, the apparatus comprising class tap extractionmeans for extracting as a class tap multiple items of information datalocated in a periphery of a target position in the first informationsignal based on the first information signal, class categorization meansfor obtaining a class code by categorizing the class tap extracted bythe class tap extraction means as any one of a plurality of classesbased on the class tap, dynamic range processing means for detecting adynamic range, which is a difference between a maximum value and aminimum value of multiple items of information data contained in theclass tap extracted by the class tap extraction means based on the classtap, to obtain area information that indicates which one of a pluralityof sub-divided areas obtained by dividing a possible area of the dynamicrange into plural ones the dynamic range belongs to, prediction tapextraction means for extracting as a prediction tap multiple items ofinformation data located in a periphery of the target position in thefirst information signal based on the first information signal,coefficient data generation means for generating first coefficient data,which is used in an estimate equation corresponding to the class code,if the dynamic range belongs to one sub-divided area, according to thearea information obtained by the dynamic range processing means and theclass code obtained by the class categorization means and for generatingsecond coefficient data, which is used in the estimate equation, if thedynamic range belongs to another sub-divided area different from the onesub-divided area, and computation means for calculate information datathat constitutes the second information signal corresponding to thetarget position in the first information signal based on the estimateequation using the first coefficient data or the second coefficient datagenerated by the coefficient data generation means and the predictiontap extracted by the prediction tap extraction means, wherein the firstcoefficient data generated by the first coefficient data generationmeans is based on a result of learning between a first learning signalthat corresponds to the first information signal and a second learningsignal that corresponds to the second information signal by use of sucha portion of the dynamic range as to belong to the one sub-divided area,and wherein the second coefficient data generated by the secondcoefficient data generation means is based on a result of learning,without the class categorization, between the first learning signal thatcorresponds to the first information signal and the second learningsignal that corresponds to the second information signal.

A method for processing an information signal according to the inventionis a method for processing an information signal that converts a firstinformation signal comprised of multiple items of information data intoa second information signal comprised of multiple items of informationdata, the method comprising a class tap extraction step of extracting asa class tap multiple items of information data located in a periphery ofa target position in the first information signal based on the firstinformation signal, a class categorization step of obtaining a classcode by categorizing the class tap extracted by the class tap extractionstep as any one of a plurality of classes based on the class tap, adynamic range processing step of detecting a dynamic range, which is adifference between a maximum value and a minimum value of multiple itemsof information data contained in the class tap extracted by the classtap extraction step based on the class tap, to obtain area informationthat indicates which one of a plurality of sub-divided areas obtained bydividing a possible area of the dynamic range into plural ones thedynamic range belongs to, a prediction tap extraction step of extractingas a prediction tap multiple items of information data located in aperiphery of the target position in the first information signal basedon the first information signal, a coefficient data generation step ofgenerating first coefficient data, which is used in an estimate equationcorresponding to the class code, if the dynamic range belongs to onesub-divided area, according to the area information obtained by thedynamic range processing step and the class code obtained by the classcategorization step and for generating second coefficient data, which isused in the estimate equation, if the dynamic range belongs to anothersub-divided area different from the one sub-divided area, and acomputation step of calculating information data that constitutes thesecond information signal corresponding to the target position in thefirst information signal based on the estimate equation using the firstcoefficient data or the second coefficient data generated by thecoefficient data generation step and the prediction tap extracted by theprediction tap extraction step.

The first coefficient data generated by the first coefficient datageneration step is based on a result of learning between a firstlearning signal that corresponds to the first information signal and asecond learning signal that corresponds to the second information signalby use of such a portion of the dynamic range as to belong to the onesub-divided area, and the second coefficient data generated by thesecond coefficient data generation step is based on a result oflearning, without the class categorization, between the first learningsignal that corresponds to the first information signal and the secondlearning signal that corresponds to the second information signal.

Further, a program related to the present invention causes a computer toperform the above-described method for processing the informationsignal. A computer-readable medium related to the present inventionrecords this program.

In the present invention, a first information signal comprised ofmultiple items of information data is converted into a secondinformation signal comprised of multiple items of information data. Itis to be noted that an information signal refers to, for example, animage signal comprised of multiple items of pixel data (sample data), anaudio signal comprised of multiple items of sample data, etc.

In this case, based on the first information signal, multiple items ofinformation data located in a periphery of the target position in thisfirst information signal are extracted as a class tap. Then, this classtap is categorized as any one of a plurality of classes, to obtain aclass code.

Further, a dynamic range is detected which is a difference between amaximum value and a minimum value of the multiple items of informationdata contained in this class tap, to obtain area information thatindicates which one of a plurality of sub-divided ranges into which apossible range of the dynamic range is divided into plural ones thisdynamic range belongs to. For example, the area information indicatingwhether a dynamic range is less than or not less than a threshold valueis obtained.

If the dynamic range belongs to one sub-divided range, first coefficientdata is generated which is used in an estimate equation that correspondsto a class code obtained by categorizing the above-described class tap.If the dynamic range belongs to another sub-divided range different fromthe one sub-divided range, on the other hand, second coefficient data isgenerated which is used in the estimate equation.

It is to be noted that the coefficient data is based on a result oflearning between a first learning signal that corresponds to the firstinformation signal and a second learning signal that corresponds to thesecond information signal by use of such a portion of the dynamic rangeas to belong to the one sub-divided range. The second coefficient data,on the other hand, is based on a result of learning, without classcategorization, between a first learning signal that corresponds to thefirst information signal and a second learning signal that correspondsto the second information signal.

For example, the first coefficient data of each class used in anestimate equation and the second coefficient data used in the estimateequation, which are obtained beforehand, are stored in the storagemeans, from which the first coefficient data that corresponds to a classindicated by a class code is read if a dynamic range belongs to onesub-divided range and the second coefficient data is read if the dynamicrange belongs to another sub-divided range different from that onesub-divided range.

Further, for example, first coefficient seed data of each class, whichis coefficient data in a production equation, which includes apredetermined parameter, for producing first coefficient data to be usedin the estimate equation, and second coefficient seed data, which iscoefficient data in the production equation for producing secondcoefficient data to be used in the estimate equation, the firstcoefficient seed data and the second coefficient seed data obtainedbeforehand, are stored in storage means, so that if a dynamic rangebelongs to one sub-divided range, the first coefficient data used in theestimate equation is produced based on the production equation usingfirst coefficient seed data that corresponds to a class indicated by theclass code stored in that storage means, and if the dynamic rangebelongs to another sub-divided range different from that one sub-dividedrange, the second coefficient data used in the estimate equation isproduced based on the production equation using the second coefficientseed data stored in the storage means.

Further, based on the first information signal, multiple items ofinformation data located in a periphery of a target position in thisfirst information signal are extracted as a prediction tap. Then, thefirst coefficient data or the second coefficient data, which is producedas described above, and this prediction tap are used to calculateinformation data, which constitutes the second information signal,corresponding to a target position of the first information signal basedon the estimate equation.

In such a manner, in the present invention, if the dynamic range belongsto one sub-divided area, the information data calculated by using thefirst coefficient data obtained so as to correspond to the class codeobtained on the basis of the class tap is output as information datathat constitutes the second information signal. In this case, the firstcoefficient data is based on a result of learning between the firstlearning signal that corresponds to the first information signal and thesecond learning signal that corresponds to the second information signalby use of such a portion of the dynamic range as to belong to the onesub-divided range, to enable the information data that constitutes thesecond information signal to be accurately obtained.

Further, in the present invention, if a dynamic range belongs to anothersub-divided range different from the one sub-divided area, informationdata calculated by using the second coefficient data is output asinformation data that constitutes the second information signal. In thiscase, the second coefficient data is based on a result of learning,without class categorization, between the first learning signal thatcorresponds to the first information signal and the second learningsignal that corresponds to the second information signal. Therefore,this second coefficient data is an average of items of the coefficientdata of the classes, so that an error of the information data thatconstitutes the second information signal, which is calculated by usingthis second coefficient data, with respect to a true value of thisinformation data is distributed in the vicinity of error 0.

It is thus, in the present invention, possible to well obtain theinformation data that constitutes the second information signal nomatter whether the dynamic range is large or small.

An apparatus for producing coefficient according to the invention is anapparatus for producing coefficient that produces coefficient data,which is used in an estimate equation to be used when converting a firstinformation signal comprised of multiple items of information data intoa second information signal comprised of multiple items of informationdata or coefficient seed data, which is coefficient data in a productionequation for producing the former coefficient data, the apparatuscomprising class tap extraction means for extracting as a class tapmultiple items of information data located in a periphery of a targetposition in a first learning signal that corresponds to the firstinformation signal based on the first learning signal, classcategorization means for obtaining a class code by categorizing theclass tap extracted by the class tap extraction means as any one of aplurality of classes based on the class tap, dynamic range processingmeans for detecting a dynamic range, which is a difference between amaximum value and a minimum value of multiple items of information datacontained in the class tap extracted by the class tap extraction meansbased on the class tap, to obtain area information that indicates whichone of a plurality of sub-divided areas obtained by dividing a possiblearea of the dynamic range into plural ones the dynamic range belongs to,prediction tap extraction means for extracting as a prediction tapmultiple items of information data located in a periphery of the targetposition in the first learning signal based on the first learning signalif the dynamic range belongs to one sub-divided area according to thearea information obtained by the dynamic range processing means, teacherdata extraction means for extracting, as teacher data, information datathat corresponds to the target position in the first learning signalbased on a second learning signal that corresponds to the secondinformation signal if the dynamic range belongs to the one sub-dividedarea according to the area information obtained by the dynamic rangeprocessing means, and computation means for obtaining the coefficientdata of each class or the coefficient seed data of each class by usingthe class code obtained by the class categorization means, theprediction tap extracted by the prediction tap extraction means, and theteacher data extracted by the teacher data extraction means.

A method for producing coefficient according to the invention is amethod for producing coefficient that produces coefficient data, whichis used in an estimate equation to be used when converting a firstinformation signal comprised of multiple items of information data intoa second information signal comprised of multiple items of informationdata or coefficient seed data, which is coefficient data in a productionequation for producing the former coefficient data, the methodcomprising a class tap extraction step of extracting as a class tapmultiple items of information data located in a periphery of a targetposition in a first learning signal that corresponds to the firstinformation signal based on the first learning signal, a class codecategorization step of obtaining a class code by categorizing the classtap extracted by the class tap extraction step as any one of a pluralityof classes based on the class tap, a dynamic range processing step ofdetecting a dynamic range, which is a difference between a maximum valueand a minimum value of multiple items of information data contained inthe class tap extracted by the class tap extraction step based on theclass tap, to obtain area information that indicates which one of aplurality of sub-divided areas obtained by dividing a possible area ofthe dynamic range into plural ones the dynamic range belongs to, aprediction tap extraction step of extracting as a prediction tapmultiple items of information data located in a periphery of the targetposition in the first learning signal based on the first learning signalif the dynamic range belongs to one sub-divided area according to thearea information obtained by the dynamic range processing step, ateacher data extraction step of extracting, as teacher data, informationdata that corresponds to the target position in the first learningsignal based on a second learning signal that corresponds to the secondinformation signal if the dynamic range belongs to the one sub-dividedarea according to the area information obtained by the dynamic rangeprocessing step, and a computation step of obtaining the coefficientdata of each class or the coefficient seed data of each class by usingthe class code obtained by the class categorization step, the predictiontap extracted by the prediction tap extraction step, and the teacherdata extracted by the teacher data extraction step.

Further, a program related to the present invention causes a computer toperform the above-described method for processing the informationsignal. A computer-readable medium related to the present inventionrecords this program.

In the present invention, the coefficient data is produced which is usedin the estimate equation used when converting a first information signalcomprised of multiple items of information data into a secondinformation signal comprised of multiple items of information data orthe coefficient seed data is produced which is coefficient data in aproduction equation for producing this coefficient data. It is to benoted that the information signal is, for example, an image signal or anaudio signal.

Multiple items of information data located in a periphery of the targetposition in this first learning signal are extracted as a class tap onthe basis of a first learning signal that corresponds to the firstinformation signal. Then, this class tap is categorized as any one of aplurality of classes, to obtain a class code.

Further, the dynamic range is detected which is a difference between amaximum value and a minimum value of the multiple items of informationdata contained in this class tap, to obtain the area information thatindicates which one of a plurality of sub-divided ranges into which apossible range of the dynamic range is divided into plural ones thisdynamic range belongs to. For example, the area information indicatingwhether a dynamic range is less than or not less than a threshold valueis obtained.

If a dynamic range belongs to one sub-divided range, multiple items ofinformation data located in a periphery of the target position in thefirst learning signal are extracted as a prediction tap based on thefirst learning signal, while information data that corresponds to thetarget position in the first learning signal is extracted as teacherdata based on a second learning signal that corresponds to the secondinformation signal.

Then, the class code thus obtained and the prediction tap and theteacher data thus extracted are used to obtain the coefficient data ofeach class or the coefficient seed data of each class. For example, foreach class, a normalization equation is produced, so that by solving thenormalization equation, the coefficient data of each class or thecoefficient seed data of each class can be calculated.

In such a manner, in the present invention, by performing learningbetween a first learning signal that corresponds to the firstinformation signal and a second learning signal that corresponds to thesecond information signal using such a portion of the dynamic range asto belong to one sub-divided range, coefficient data of each class orcoefficient seed data of each class is obtained, thereby enabling wellto be obtained the coefficient data that is used in the above-describedapparatus for processing the information signal etc.

An apparatus for producing a lookup table according to the invention isan apparatus for producing a lookup table that produces a correspondencerelationship between a first class code and a second class code, whichare used when converting a first information signal comprised ofmultiple items of information data into a second information signalcomprised of multiple items of information data, the apparatuscomprising class tap extraction means for extracting as a class tapmultiple items of information data located in a periphery of a targetposition in a first learning signal that corresponds to the firstinformation signal based on the first learning signal, classcategorization means for obtaining a class code by categorizing theclass tap extracted by the class tap extraction means as any class of aplurality of class taps based on the class tap, dynamic range processingmeans for detecting a dynamic range which is a difference between amaximum value and a minimum value of multiple items of information datacontained in the class tap extracted by the class tap extraction meansbased on the class tap, to obtain area information that indicates whichone of a plurality of sub-divided areas obtained by dividing a possiblearea of the dynamic range into plural ones the dynamic range belongs to,prediction tap extraction means for extracting as a prediction tapmultiple items of information data located in a periphery of the targetposition in the first learning signal based on the first learningsignal, first coefficient data generation means for generatingcoefficient data, which is used in an estimate equation at a class thatcorresponds to a class code obtained by the class categorization means,predictive computation means for calculating information data thatcorresponds to a target position in the first learning signal based onthe estimate equation using the coefficient data generated by the firstcoefficient data generation means and the prediction tap extracted bythe prediction tap extraction means, second coefficient data generationmeans for generating coefficient data, which is used in an estimateequation at the plurality of classes, all-the-class predictivecomputation means for calculating information data that corresponds to atarget position in the first learning signal for each of the classesbased on the estimate equation using coefficient data of each classgenerated by the second coefficient data generation means and aprediction tap extracted by the prediction tap extraction means, teacherdata extraction means for extracting, as teacher data, information datathat corresponds to a target position in the first learning signal basedon a second learning signal that corresponds to the second informationsignal, error calculation means for calculating an error of theinformation data obtained by the predictive computation means withrespect to the teacher data extracted by the teacher data extractionmeans, all-the-class error calculation means for calculating an error ofinformation data of each of the classes obtained by the all-the-classpredictive computation means with respect to the teacher data extractedby the teacher data extraction means, error addition means for adding anerror obtained by the error calculation means to an error of each of theclasses obtained by the all-the-class calculation means to obtain anerror sum of the classes, error sum accumulation means for adding avalue that corresponds to a magnitude of the error sum of each of theclasses obtained by the error addition means to an accumulated value ofeach output class at an input class that corresponds to the class codeobtained by the class categorization means if the dynamic range belongsto another sub-divided area different from that one sub-divided areaaccording to the area information obtained by the dynamic rangeprocessing means, and table production means for allocating an outputclass in which an accumulated value of each output class to each of theinput classes, the accumulated value being obtained by the error sumaccumulation means, is minimized based on the accumulated value at eachof the input classes, to produce a correspondence relationship betweenthe first class code that corresponds to the input class and the secondclass code that corresponds to the output class. The items of thecoefficient data generated by the first coefficient data generationmeans and generated by the second coefficient data generation means arebased on a result of learning between a first learning signal thatcorresponds to the first information signal and a second learning signalthat corresponds to the second information signal by use of only such aportion of the dynamic range as to belong to the one sub-divided area.

A method for producing a lookup table according to the invention is amethod for producing a lookup table that produces a correspondencerelationship between a first class code and a second class code, whichare used when converting a first information signal comprised ofmultiple items of information data into a second information signalcomprised of multiple items of information data, the method comprising aclass tap extraction step of extracting as a class tap multiple items ofinformation data located in a periphery of a target position in a firstlearning signal that corresponds to the first information signal basedon the first learning signal, a class categorization step of obtaining aclass code by categorizing the class tap extracted by the class tapextraction step as any class of a plurality of class taps based on theclass tap, a dynamic range processing step of detecting a dynamic rangewhich is a difference between a maximum value and a minimum value ofmultiple items of information data contained in the class tap extractedby the class tap extraction step based on the class tap, to obtain areainformation that indicates which one of a plurality of sub-divided areasobtained by dividing a possible area of the dynamic range into pluralones the dynamic range belongs to, a prediction tap extraction step ofextracting as a prediction tap multiple items of information datalocated in a periphery of the target position in the first learningsignal based on the first learning signal, a first coefficient datageneration step of generating coefficient data, which is used in anestimate equation at a class that corresponds to a class code obtainedby the class categorization step, a predictive computation step ofcalculating information data that corresponds to a target position inthe first learning signal based on the estimate equation usingcoefficient data generated by the first coefficient data generation stepand the prediction tap extracted by the prediction tap extraction step,a second coefficient data generation step of generating coefficientdata, which is used in an estimate equation at the plurality of classes,an all-the-class predictive computation step of calculating informationdata that corresponds to a target position in the first learning signalfor each of the classes based on the estimate equation using coefficientdata of each class generated by the second coefficient data generationstep and a prediction tap extracted by the prediction tap extractionstep, a teacher data extraction step of extracting, as teacher data,information data that corresponds to a target position in the firstlearning signal based on a second learning signal that corresponds tothe second information signal, an error calculation step of calculatingan error of the information data obtained by the predictive computationstep with respect to the teacher data extracted by the teacher dataextraction step, an all-the-class error calculation step of calculatingan error of information data of each of the classes obtained by theall-the-class predictive computation step with respect to teacher dataextracted by the teacher data extraction step, an error addition step ofadding an error obtained by the error calculation step to an error ofeach of the classes obtained by the all-the-class calculation step toobtain an error sum of the classes, an error sum accumulation step ofadding a value that corresponds to a magnitude of an error sum of eachof the classes obtained by the error addition step to an accumulatedvalue of each output class at an input class that corresponds to theclass code obtained by the class categorization step if the dynamicrange belongs to another sub-divided area different from that onesub-divided area according to the area information obtained by thedynamic range processing step, and a table production step of allocatingan output class in which an accumulated value of each output class toeach of the input classes, the accumulated value being obtained by theerror sum accumulation step, is minimized based on the accumulated valueat each of the input classes, to produce a correspondence relationshipbetween the first class code that corresponds to the input class and thesecond class code that corresponds to the output class. The items of thecoefficient data generated by the first coefficient data generation stepand generated by the second coefficient data generation step are basedon a result of learning between a first learning signal that correspondsto the first information signal and a second learning signal thatcorresponds to the second information signal by use of only such aportion of the dynamic range as to belong to the one sub-divided area.

Further, a program related to the present invention causes a computer toperform the above-described method for processing the informationsignal. A computer-readable medium related to the present inventionrecords this program.

In the present invention, a correspondence relationship is producedbetween a first class code and a second class code, which is used whenconverting a first information signal comprised of multiple items ofinformation data into a second information signal comprised of multipleitems of information data. It is to be noted that the information signalis, for example, an image signal or an audio signal.

Based on the first learning signal that corresponds to the firstinformation signal, multiple items of information data located in aperiphery of the target position in this first learning signal areextracted as a class tap. Then, this class tap is categorized as any oneof a plurality of classes, to obtain a class code.

Further, the dynamic range is detected which is a difference between amaximum value and a minimum value of the multiple items of informationdata contained in this class tap, to obtain area information thatindicates which one of a plurality of sub-divided ranges into which apossible range of the dynamic range is divided into plural ones thisdynamic range belongs to. For example, the area information indicatingwhether a dynamic range is less than or not less than a threshold valueis obtained.

Further, based on a student signal, multiple items of information datalocated in a periphery of the target position in this first learningsignal is extracted as a prediction tap. Further, coefficient data isproduced which is used in an estimate equation at a class thatcorresponds to the class code obtained by categorizing class taps asdescribed above. Then, these coefficient data and prediction tap areused to calculate information data that corresponds to a target positionin the first learning signal based on the estimate equation.

Further, coefficient data is produced which is used in an estimateequation at a plurality of classes. Then, the coefficient data of eachclass and the prediction tap are used to calculate information data thatcorresponds to a target position in the first learning signal for eachclass based on the estimate equation.

It is to be noted that the coefficient data generated as described aboveis based on a result of learning between a first learning signal thatcorresponds to the first information signal and a second learning signalthat corresponds to the second information signal by use only of such aportion of a dynamic range as to belong to one sub-divided range.

Further, information data that corresponds to a target position in thefirst learning signal is extracted as teacher data based on the secondlearning signal that corresponds to the second information signal. Then,with respect to this teacher data, an error is computed of informationdata that corresponds to a class code calculated as described above.Furthermore, with respect to this teacher data, an error is computed ofinformation data of each class calculated as described above.

Further, the error for the class code and the error for each class areadded to obtain an error sum for each class. Then, if a dynamic rangebelongs to another sub-divided range different from the one sub-dividedrange, a value that corresponds to a value of the error sum for eachclass is added to an accumulated value for each of the output classes atan input class that corresponds to a class code.

Then, based on the accumulated value for each of the output classes ateach input class, such an output class as to minimize the accumulatedvalue is allocated to each of the input classes, to produce acorrespondence relationship between a first class code that correspondsto the input class and a second class code that corresponds to theoutput class.

In such a manner, in the present invention, if a dynamic range belongsto another range different from that one range, it is possible to wellproduce such a lookup table that a first class code can be convertedinto a second class code in such a manner that an addition mean value ofinformation data calculated by using coefficient data that correspondsto the first class code and information data calculated by usingcoefficient data that corresponds to the second class code may mostapproach to a true value of information data that constitutes the secondinformation signal.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a configuration of an apparatus forprocessing an image signal according to an embodiment;

FIG. 2 is a block diagram showing a positional relationship of pixelsbetween an SD signal and an HD signal;

FIG. 3A is a block diagram showing one example of a class tap;

FIG. 3B is a block diagram showing one example of a prediction tap;

FIG. 4 is a block diagram showing a configuration of a lookup table;

FIG. 5 is a block diagram showing a position of a prediction tap;

FIG. 6 is a block diagram showing a configuration of an apparatus forproducing coefficient data;

FIG. 7 is a block diagram showing a configuration of an apparatus forproducing a lookup table;

FIG. 8 is a diagram illustrating accumulation processing;

FIG. 9 is a diagram illustrating output class allocation processing;

FIG. 10 is a block diagram showing a configuration of an apparatus forprocessing the image signal with software implementation;

FIG. 11 is a flowchart showing image signal processing;

FIG. 12 is a flowchart showing coefficient data production processing;

FIG. 13 is a flowchart showing lookup table production processing;

FIG. 14 is a block diagram showing a configuration of an apparatus forprocessing an image signal according to another embodiment;

FIG. 15 is a flowchart showing image signal processing;

FIG. 16 is a block diagram showing a configuration of an apparatus forprocessing an image signal according to a further embodiment;

FIG. 17 is a diagram illustrating a method for producing coefficientseed data;

FIG. 18 is a block diagram showing a configuration of an apparatus forproducing coefficient seed data; and

FIG. 19 is a block diagram showing a configuration of an apparatus forproducing a lookup table.

BEST MODE FOR CARRYING OUT THE INVENTION

The following will describe embodiments of the present invention withreference to drawings. FIG. 1 shows a configuration of an apparatus 100for processing an image signal according to an embodiment. Thisapparatus 100 for processing the image signal converts an image signalhaving a low resolution or standard resolution (hereinafter referred toas “standard definition (SD) signal”) into an image signal having highresolution (hereinafter referred to as “high definition (HD) signal”).It is to be noted that the SD signal constitutes a first informationsignal and the HD signal constitutes a second information signal.

FIG. 2 shows a positional relationship of pixels between an SD signaland an HD signal. ◯ indicates a pixel position of the SD signal and “X”,a pixel position of the HD signal. In this case, to one pixel of the SDsignal, four pixels of the HD signal correspond. That is, in the presentembodiment, an SD signal is converted into an HD signal having twice thenumbers of vertical and horizontal pixels, respectively. In the presentembodiment, each of the SD and HD signals is comprised of multiple itemsof pixel data of 8-bit.

Referring back to FIG. 1, the apparatus 100 for processing the imagesignal comprises an input terminal 101 to which an SD signal is inputand a class tap extraction circuit 102 for extracting as a class tap aplurality of pixel data pieces located in a periphery of a targetposition in an SD signal input to this input terminal 101, based on thisSD signal. In the present embodiment, for example, as shown in FIG. 3A,seven SD pixel data pieces located in a periphery of a target positionin the SD signal are extracted as a class tap.

The apparatus 100 for processing the image signal further comprises aclass categorization circuit 103 for categorizing a class tap extractedby the class tap extraction circuit 102 as any one of a plurality ofclasses based on this class tap to obtain a class code Ca indicative ofa class of this class tap. This class categorization is performed byutilizing any compression processing such as adaptive dynamic rangecoding (ADRC), differential pulse code modulation (DPCM) (in predictioncoding), or vector quantization (VQ).

The following will describe a case of performing K-bit ADRC. In theK-bit ADRC, a dynamic range DR=MAX−MIN is detected, which is adifference between a maximum value MAX and a minimum value MIN of itemsof pixel data included in a class tap, and based on this dynamic rangeDR, each of the items of pixel data included in the class tap isre-quantized into K bits.

That is, each of the items of pixel data included in a class tap issubtracted by a minimum value MIN and a resultant remainder is divided(quantized) by DR/2^(K). In such a manner, each of the items of pixeldata that constitute the class tap is re-quantized into K bits, whichare arranged in predetermined order into a bit string that is thenoutput as a class code Ca.

Therefore, in 1-bit ADRC, each of the items of pixel data included inthis class tap is subtracted by a minimum value MIN and a resultantremainder is divided by DR/2. Accordingly, each of the items of pixeldata included in the class tap is re-quantized into one bit, which arearranged in predetermined order into a bit string that is then output asa class code Ca.

The apparatus 100 for processing the image signal further comprises adynamic range processing circuit (DR processing circuit) 104. This DRprocessing circuit 104 detects a dynamic range DR=MAX−MIN, which is adifference between a maximum value MAX and a minimum value MIN of itemsof pixel data included in a class tap extracted by the class tapextraction circuit 102, to acquire an area information AR that indicateswhich one of a plurality of sub-divided areas obtained by dividing apossible area of this dynamic range DR into plural ones it belongs to.

In the present embodiment, the possible coverage area of the dynamicrange DR is divided by two by using a predetermined threshold value Th.If DR≧Th, that is, if the dynamic range DR is not less than thethreshold value Th, the DR processing circuit 104 outputs “0” as areainformation AR, while if DR<Th, that is, if the dynamic range DR is lessthan the threshold value Th, it outputs “1” as the area information AR.

The apparatus 100 for processing the image signal further comprises alookup table (LUT) 105 as class code conversion means for converting aclass code Ca obtained by the class categorization circuit 103 into aclass code Cb that corresponds to this class code Ca. This lookup table105 is controlled in its operation based on the area information ARobtained by the DR processing circuit 104. That is, this lookup table105 becomes active only when the area information AR is “1”, to outputthe class code Cb that corresponds to the class code Ca. FIG. 4 shows aconfiguration of the lookup table 105, which stores a correspondencerelationship between the class code Ca and the class code Cb.

This lookup table 105 converts a class code Ca into a class code Cb insuch a manner that an addition mean value of items of the pixel datacalculated by using coefficient data Wi that corresponds to the classcode Ca and the pixel data calculated by using coefficient data Wi thatcorresponds to the class code Cb may most approach a true value of pixeldata that constitutes an HD signal. How to produce this lookup table 105will be described later.

The apparatus 100 for processing the image signal further comprises acoefficient memory 106. This coefficient memory 106 stores coefficientdata Wi of each class, which is used in an estimate equation used bylater-described predictive computation circuits 108 a and 108 b. Thiscoefficient data Wi is information used to convert an SD signal into anHD signal.

As described above, to convert the SD signal into the HD signal, it isnecessary to obtain four pixels (y₁ to y₄) of the HD signal for onepixel (x₀) of the SD signal. In this case, the four pixels of the HDsignal have different shifts in phase with respect to the one pixel ofthe corresponding SD signal. Therefore, the coefficient memory 106stores coefficient data Wi for each combination of classes and positionsof output pixels (positions of y₁ to y₄).

Coefficient data Wi stored in this coefficient memory 106 has beenobtained by performing learning between a student signal, which is afirst learning signal that corresponds to an SD signal, and a teachersignal, which is a second learning signal that corresponds to an HDsignal, by use of such a portion of the dynamic range as to satisfy arelationship of DR≧Th. How to produce this coefficient data Wi will bedescribed later.

The coefficient memory 106 is supplied as read address information theclass code Ca output from the above-described class categorizationcircuit 103 and the class code Cb output from the above-described lookuptable 105. The coefficient memory 106 outputs coefficient data Wi-a of aclass indicated by the class code Ca and coefficient data Wi-b of aclass indicated by the class code Cb. This coefficient memory 106constitutes first coefficient data generation means and secondcoefficient data generation means.

The apparatus 100 for processing the image signal further comprises aprediction tap extraction circuit 107 for extracting as a prediction tapmultiple items of pixel data located in a periphery of a target positionin the SD signal input to the input terminal 101 based on this SDsignal. In the present embodiment, for example, as shown in FIG. 3B, 13items of SD pixel data located in a periphery of a target position in anSD signal are extracted as a prediction tap. FIG. 5 indicates a tapposition of this prediction tap.

The apparatus 100 for processing the image signal further comprisespredictive computation circuits 108 a and 108 b as first and secondcomputation means, respectively. These predictive computation circuits108 a and 108 b obtain items of pixel data y_(1-a) to y_(4-a) andy_(1-b) to y_(4-b) based on an estimate equation (1) from pixel data xiextracted as prediction tap by the prediction tap extraction circuit 107and items of coefficient data Wi-a and Wi-b output from the coefficientmemory 106, respectively. In this Equation (1), n indicates the numberof items of the pixel data that constitute the prediction tap and n=13in the present embodiment. It is to be noted that one computationcircuit may be adapted to serve as both of the prediction computationcircuits 108 a and 108 b.

$\begin{matrix}{Y = {\sum\limits_{i = 1}^{n}{{Wi} \cdot {xi}}}} & {{Equation}\mspace{14mu}(1)}\end{matrix}$

As described above, to convert an SD signal into an HD signal, it isnecessary to obtain four pixels of the HD signal for one pixel of the SDsignal. Therefore, these predictive computation circuits 108 a and 108 beach produce four items of pixel data y₁ to y₄ for each target positionin the SD signal.

That is, these predictive computation circuits 108 a and 108 b aresupplied with pixel data xi as prediction tap that corresponds to atarget position in the SD signal from the prediction tap extractioncircuit 107 and items of coefficient data Wi-a and Wi-b of theabove-described four output pixels from the coefficient memory 106,respectively. Then, these predictive computation circuits 108 a and 108b obtain four items of pixel data y_(1-a) to y_(4-a) and y_(1-b) toy_(4-b), respectively and individually, by the above-described Equation(1).

The apparatus 100 for processing the image signal further comprises anadder 109. This adder 109 is controlled in its operation based on thearea information AR obtained by the DR processing circuit 104. That is,the adder 109 outputs four items of pixel data y_(1-a) to y_(4-a)obtained by the predictive computation circuit 108 a as four items ofpixel data y₁ to y₄ that constitute the HD signal if the areainformation AR is “0” and the dynamic range DR is not less than thethreshold value Th.

If the area information AR is “1” and the dynamic range DR is less thanthe threshold value Th, on the other hand, the adder 109 outputs anaddition mean value (y_(1-a)+y_(1-b))/2 through (y_(4-a)+y_(4-b))/2 offour items of pixel data y_(1-n) to y_(4-n) obtained by the predictivecomputation circuit 108 a and four items of pixel data y_(1-b) toy_(4-b) obtained by the predictive computation circuit 108 b as fouritems of pixel data y₁ to y₄ that constitute the HD signal.

The apparatus 100 for processing the image signal further comprises apost-processing circuit 110 for obtaining the HD signal bylinear-serializing four items of pixel data y₁ to y₄ that constitute theHD signal, which correspond to each of the target positions in the SDsignal, serially output by the adder 109 and an output terminal 111 foroutputting this HD signal.

The following will describe operations of the apparatus 100 forprocessing the image signal as shown in FIG. 1.

An SD signal input to the input terminal 101 is supplied to the classtap extraction circuit 102. This class tap extraction circuit 102extracts as a class tap multiple items of pixel data located in aperiphery of a target position in the SD signal based on this SD signal(see FIG. 3A). This class tap is supplied to the class categorizationcircuit 103 and the DR processing circuit 104.

The class categorization circuit 103 performs data compressionprocessing such as ADRC processing on each of the items of pixel datacontained in a class tap, to obtain a class code Ca as a first classcode that indicates a class of this class tap. This class code Ca issupplied as read address information to the lookup table 105 and thecoefficient memory 106.

Further, the DR processing circuit 104 detects dynamic range DR=MAX−MIN,which is a difference between a maximum value MAX and a minimum valueMIN of the items of pixel data contained in the class tap and, if thisdynamic range DR is not less than a threshold value Th, outputs “0” asarea information AR and, otherwise, outputs “1” as the area informationAR. This area information AR is supplied as an operation control signalto the lookup table 105 and the adder 109.

The lookup table 105 becomes active if the area information AR is “1”,that is, the dynamic range DR is less than the threshold value Th, tooutput a class code Cb as a second class code that corresponds to theclass code Ca obtained by the class categorization circuit 103. Thisclass code Cb is supplied as read address information to the coefficientmemory 106.

When the class code Ca is supplied to the coefficient memory 106 as readaddress information, coefficient data Wi-a of four output pixels thatcorresponds to a class indicated by the class code Ca is read from thiscoefficient memory 106 and supplied to the predictive computationcircuit 108 a. Similarly, when the class code Cb is supplied to thecoefficient memory 106 as read address information, coefficient dataWi-b of four output pixels that corresponds to a class indicated by theclass code Cb is read from this coefficient memory 106 and supplied tothe predictive computation circuit 108 b.

Further, the SD signal input to the input terminal 101 is supplied tothe predictive tap extraction circuit 107. This predictive tapextraction circuit 107 extracts as a prediction tap multiple items ofpixel data located in a periphery of a target position in the SD signalbased on this SD signal (see FIG. 3B). Pixel data xi as this predictiontap is supplied to the predictive computation circuits 108 a and 108 b.

The predictive computation circuit 108 a uses the pixel data xi and thecoefficient data Wi-a to calculate four items of pixel data y_(1-a) toy_(4-a) that correspond to the target position in the SD signal based onthe above-described Equation (1). Similarly, the predictive computationcircuit 108 b uses the pixel data xi and the coefficient data Wi-b tocalculate four items of pixel data y_(1-b) to y_(4-b) that correspond tothe target position in the SD signal based on the above-describedEquation (1). The items of pixel data y_(1-a) to y_(4-a) and y_(1-b) toy_(4-b) calculated by these predictive computation circuits 108 a and108 b, respectively, are supplied to the adder 109.

The adder 109 outputs four items of pixel data y_(1-a) to y_(4-a)obtained by the predictive computation circuit 108 a as four items ofpixel data y₁ to y₄ that constitute the HD signal if the areainformation AR is “0” and the dynamic range DR is not less than thethreshold value Th. If the area information AR is “1” and the dynamicrange DR is less than the threshold value Th, on the other hand, theadder 109 outputs an addition mean value (y_(1-b)+y_(1-b))/2 through(y_(4-a)+y_(4-b))/2 of four items of pixel data y_(1-a) to y_(4-a)obtained by the predictive computation circuit 108 a and four items ofpixel data y_(1-b) to y_(4-b) obtained by the predictive computationcircuit 108 b as four items of pixel data y₁ to y₄ that constitute theHD signal.

The four items of pixel data y₁ to y₄ that constitute the HD signal andare serially output from this adder 109 are supplied to thepost-processing circuit 110. This post-processing circuit 110 obtains anHD signal by linear-serializing four items of pixel data y₁ to y₄, whichconstitute the HD signal and are serially supplied from the adder 109,corresponding to each of the target positions in the SD signal. This HDsignal is output to the output terminal 111.

If the area information AR is “0”, that is, if the dynamic range DR isnot less than the threshold value Th, the above-described apparatus 100for processing the image signal outputs items of pixel data y_(1-a) toy_(4-a) calculated by using coefficient data Wi-a obtained so as tocorrespond to the class code Ca as items of pixel data y₁ to y₄ thatconstitute the HD signal.

In this case, as described above, the coefficient data Wi-a has beenobtained by performing learning between a student signal (first learningsignal) that corresponds to the SD signal and a teacher signal (secondlearning signal) that corresponds to the HD signal by using such aportion of the dynamic range DR as to be not less than the thresholdvalue Th, thereby enabling accurately to be obtained items of pixel datay₁ to y₄ that constitute the HD signal.

Further, in the above-described apparatus 100 for processing the imagesignal, if the area information AR is “1”, that is, the dynamic range DRis less than the threshold value Th, an addition mean value(y_(1-a)+y_(1-b))/2 through (y_(4-a)+y_(4-b))/2 of items of pixel datay_(1-a) to y_(4-a) obtained by using the coefficient data Wi-a obtainedso as to correspond to the class code Ca and items of pixel data y_(1-b)to y_(4-b) calculated by using the coefficient data Wi-b obtained so asto correspond to the class code Cb obtained by converting this classcode Ca is output as items of pixel data y₁ to y₄ that constitute the HDsignal.

In this case, the class code Ca is converted into the class code Cb insuch a manner that the above addition mean value (y_(1-a)+y_(1-b))/2through (y_(4-a)+y_(4-b))/2 may most approach a true value of the itemsof pixel data y₁ to y₄ that constitute the HD signal, thereby enablingaccurately to be obtained the items of pixel data y₁ to y₄ thatconstitute the HD signal.

Therefore, by the above-described apparatus 100 for processing the imagesignal, it is possible to well obtain items of pixel data y₁ to y₄ thatconstitute the HD signal no matter whether the dynamic range DR is largeor small. This apparatus 100 for processing the image signal can beapplied to an apparatus for outputting an image signal, etc., forexample, a TV receiver and an image signal reproduction apparatus.

The following will describe a method of producing coefficient data Wi(i=1 to n) of each class, which is stored in the coefficient memory 106.This coefficient data Wi is produced by learning.

A learning method will be described below. In the above-describedEquation (1), before learning, items of coefficient data W₁, W₂, W_(n)are undetermined coefficients. Learning is performed on multiple itemsof signal data for each class. If the number of items of learning datais m, the following Equation (2) is established according to Equation(1). n indicates the number of prediction taps.y _(k) =W ₁ ×x _(k1) +W ₂ ×x _(k2) + . . . +W _(n) ×x _(kn)(k=1, 2, . .. , m)  Equation (2)

If m>n, items of the coefficient data W₁, W₂, . . . , W_(n) are notdetermined uniquely, so that an element e_(k) of an error vector e isdefined in the following Equation (3), to obtain coefficient data bywhich e² in Equation (4) is minimized. The coefficient data is obtaineduniquely by the so-called least-squares method.e _(k) =y _(k) ={W ₁ ×x _(k1) +W ₂ ×x _(k2) + . . . +W _(n) ×w_(kn)}(k=1, 2, . . . m)  Equation (3)

$\begin{matrix}{e^{2} = {\sum\limits_{k = 1}^{m}{e_{k}}^{2}}} & {{Equation}\mspace{14mu}(4)}\end{matrix}$

By an actual calculation method of obtaining the coefficient data bywhich e² in Equation (4) is minimized, first, as shown in Equation (5),e² can be partial-differentiated by using coefficient data Wi (i=1 ton), to obtain such coefficient data Wi as to reduce thepartial-differentiated value to 0.

$\begin{matrix}{\frac{\partial e^{2}}{\partial{Wi}} = {{\sum\limits_{k = 1}^{m}{2\left( \frac{\partial{ek}}{\partial{Wi}} \right)e_{k}}} = {\sum\limits_{k = 1}^{m}{2{x_{ki} \cdot e_{k}}}}}} & {{Equation}\mspace{14mu}(5)}\end{matrix}$

By defining x_(ji) and y_(i) as given in Equations (6) and (7), theEquation (5) can be written in a form of a determinant of Equation (8).This Equation (8) is a normal equation for calculating coefficient data.By solving this normal equation by a generic solution such as asweeping-out (Gauss-Jordan elimination) method, items of the coefficientdata Wi (i=1 to n) can be obtained.

$\begin{matrix}{X_{ji} = {\sum\limits_{p = 1}^{m}{{xpi} \cdot {xpj}}}} & {{Equation}\mspace{14mu}(6)}\end{matrix}$

$\begin{matrix}{{Yi} = {\sum\limits_{k = 1}^{m}{{xki} \cdot {yk}}}} & {{Equation}\mspace{14mu}(7)}\end{matrix}$

$\begin{matrix}{{\begin{bmatrix}X_{11} & X_{12} & \cdots & X_{1n} \\X_{21} & X_{22} & \cdots & X_{2n} \\\cdots & \cdots & \cdots & \cdots \\X_{n\; 1} & X_{n\; 2} & \cdots & X_{nn}\end{bmatrix}\begin{bmatrix}W_{1} \\W_{2} \\\cdots \\W_{n}\end{bmatrix}} = \begin{bmatrix}Y_{1} \\Y_{2} \\\cdots \\Y_{n}\end{bmatrix}} & {{Equation}\mspace{14mu}(8)}\end{matrix}$

FIG. 6 shows a configuration of an apparatus 200 for producingcoefficient data that produces coefficient data Wi to be stored in thecoefficient memory 106 shown in FIG. 1.

This apparatus 200 for producing the coefficient data comprises an inputterminal 201 to which an HD signal is input as a teacher signal, whichis a second learning signal, and an SD signal production circuit 202 forhorizontal and vertical thinning processing on this HD signal to therebyobtain an SD signal as a student signal, which is a first learningsignal.

The apparatus 200 for producing the coefficient data further comprises aclass tap extraction circuit 203 for extracting as a class tap multipleitems of pixel data located in a periphery of a target position in theSD signal output from the SD signal production circuit 202 based on thisSD signal. This class tap extraction circuit 203 is configured similarto the class tap extraction circuit 102 in the above-described apparatus100 for processing the image signal shown in FIG. 1.

The apparatus 200 for producing the coefficient data further comprises aclass categorization circuit 204 for categorizing a class tap extractedby the class tap extraction circuit 203 into any one of a plurality ofclasses based on this class tap, to obtain a class code Ca indicative ofa class of this class tap. This class categorization circuit 204 isconfigured similar to the class categorization circuit 103 in theabove-described apparatus 100 for processing the image signal shown inFIG. 1.

The apparatus 200 for producing the coefficient data further comprises adynamic range processing circuit (DR processing circuit) 205. This DRprocessing circuit 205 is configured similar to the DR processingcircuit 104 in the above-described apparatus 100 for processing theimage signal shown in FIG. 1 and, if DR≧Th, outputs “0” as areainformation AR and, otherwise, outputs “1” as the area information AR.

The apparatus 200 for producing the coefficient data further comprises adelay circuit 206 for time-adjusting an SD signal output from the SDsignal production circuit 202 and a prediction tap extraction circuit207 for extracting as a prediction tap multiple items of pixel datalocated in a periphery of a target position in this SD signal outputfrom this delay circuit 206 based on this SD signal. This prediction tapextraction circuit 207 is configured similar to the prediction tapextraction circuit 107 in the above-described apparatus 100 forprocessing the image signal shown in FIG. 1.

This prediction tap extraction circuit 207 is controlled in itsoperation based on area information obtained by the DR processingcircuit 205. That is, the prediction tap extraction circuit 207 extractsa prediction tap if the area information AR is “0”, that is, if adynamic range DR is not less than a threshold value Th, and does notextract it if the area information is “1”, that is, if the dynamic rangeDR is less than the threshold value Th.

The apparatus 200 for producing the coefficient data further comprises adelay circuit 208 for time-adjusting the HD signal input to the inputterminal 201 and a teacher data extraction circuit 209 for extracting asteacher data four items of pixel data that constitute an HD signalcorresponding to a target position in an SD signal based on the HDsignal output from this delay circuit 208.

This teacher data extraction circuit 209 is also controlled in itsoperation based on the area information obtained by the DR processingcircuit 205. That is, the teacher data extraction circuit 209 extractsteacher data if the area information AR is “0”, that is, if the dynamicrange DR is not less than the threshold value Th, while it does notextract the teacher data if the area information AR is “1”, that is, ifthe dynamic range DR is less than the threshold value Th.

In this case, corresponding to a target position in the SD signal, onelearning pair data is configured between a prediction tap extracted bythe prediction tap extraction circuit 207 and teacher data extracted bythe teacher data extraction circuit 209. As described above, only if thedynamic range DR is not less than the threshold value Th, a predictiontap and teacher data are extracted as described above to thereby performlearning by using only such a portion of the dynamic range DR as to benot less than the threshold value Th.

The apparatus 200 for producing the coefficient data further comprises alearning pair storage section 210. This learning pair storage section210 stores, for each class, as learning pair data a prediction tap andteacher data that are extracted respectively by the prediction tapextraction circuit 207 and the teacher data extraction circuit 209corresponding to each target position in the SD signal based on a classcode Ca obtained by the class categorization circuit 204.

The apparatus 200 for producing the coefficient data further comprises acomputation circuit 211 for obtaining coefficient data Wi of each class.This computation circuit 211 uses multiple items of learning pair datastored in the learning pair storage section 210, to thereby produce anormal equation (see Equation (8)) for calculating coefficient data Wifor each class. It is to be noted that in this case, the computationcircuit 211 produces a normal equation for each output pixel position(position of y₁ to y₄). That is, the computation circuit 211 produces anormal equation for each combination of classes and output pixelpositions. Further, this computation circuit 211 calculates coefficientdata Wi for each combination of classes and output pixel positions bysolving each normal equation.

The apparatus 200 for producing the coefficient data further comprises acoefficient memory 212 for storing coefficient data Wi obtained by thecomputation circuit 211.

The following will describe operations of the apparatus 200 forproducing the coefficient data shown in FIG. 6.

An HD signal as a teacher signal is input to the input terminal 201. TheSD signal production circuit 202 performs horizontal and verticalthinning processing on this HD signal, to produce an SD signal as astudent signal.

The SD signal obtained by the SD signal production circuit 202 issupplied to the class tap extraction circuit 203. This class tapextraction circuit 203, based on the SD signal, extracts as a class tapmultiple items of pixel data located in a periphery of a target positionin this SD signal (See FIG. 3A). This class tap is supplied to the classcategorization circuit 204 and the DR processing circuit 205.

The class categorization circuit 204 performs data compressionprocessing such as ADRC processing on each of the items of pixel datacontained in the class tap, to obtain a class code Ca indicative of aclass of this class tap. This class code Ca is supplied to the learningpair storage section 210.

Further, the DR processing circuit 205 detects dynamic range=MAX−MIN,which is a difference between a maximum value MAX and a minimum valueMIN of the items of pixel data contained in the class tap, and outputs“0” as area information if this dynamic range DR is not less than athreshold value Th while if this dynamic range DR is less than thethreshold value Th, it outputs “1” as the area information AR. This areainformation AR is supplied as the operation control signal to theprediction tap extraction circuit 207 and the teacher data extractioncircuit 209.

Further, the SD signal obtained by the SD signal production circuit 202is time-adjusted by the delay circuit 206 and then supplied to theprediction tap extraction circuit 207. Only if the dynamic range DR isnot less than the threshold value Th, this prediction tap extractioncircuit 207, based on the SD signal, extracts as a prediction tapmultiple items of pixel data located in a periphery of a target positionin this SD signal (see FIG. 3B). This prediction tap is supplied to thelearning pair storage section 210.

Further, the HD signal input to the input terminal 201 is time-adjustedby the delay circuit 208 and then supplied to the teacher dataextraction circuit 209. Only if the dynamic range DR is not less thanthe threshold value Th, this teacher data extraction circuit 209, basedon the HD signal, extracts as teacher data four items of pixel data thatconstitute the HD signal corresponding to the target position in the SDsignal. This teacher data is supplied to the learning pair storagesection 210.

Based on the class code Ca obtained by the class categorization circuit204, the learning pair storage section 210 stores as learning pair dataa prediction tap and teacher data extracted respectively by theprediction tap extraction circuit 207 and the teacher data extractioncircuit 209 corresponding to each target position in the SD signal, foreach class.

Then, the computation circuit 211 uses multiple items of learning pairdata stored in the learning pair storage section 210, to thereby producea normal equation (see Equation (8)) for calculating coefficient data Wifor each combination of classes and output pixel positions. Furthermore,this computation circuit 211 solves each normal equation to calculatecoefficient data Wi for each combination of the classes and the outputpixel positions. The coefficient data Wi thus obtained by thecomputation circuit 211 is stored in a coefficient memory 212.

In such a manner, in the apparatus 200 for producing the coefficientdata shown in FIG. 6, it is possible to produce coefficient data Wi,which is used in an estimate equation, for each combination of classesand output pixel positions (positions of y₁ to y₄) and which is to bestored in the coefficient memory 106 in the apparatus 100 for processingthe image signal shown in FIG. 1. In this case, only if the dynamicrange DR is not less than a threshold value Th, a prediction tap andteacher data are extracted to obtain learning pair data as describedabove, so that coefficient data Wi to be produced is based on a resultof learning between a student signal (SD signal) and a teacher signal(HD signal) by use of such a portion of the dynamic range as to be notless than the threshold value Th.

The following will describe a method of producing a correspondencerelationship between a class code Ca and a class code Cb which arestored in the lookup table 105. FIG. 7 shows a configuration of anapparatus for producing a lookup table (LUT production apparatus 300)that produces the correspondence relationship.

This LUT production apparatus 300 comprises an input terminal 301 towhich an HD signal as a teacher signal, which is a second learningsignal, is input and an SD signal production circuit 302 for performinghorizontal and vertical thinning processing on this HD signal to therebyobtain an SD signal as a student signal, which is a first learningsignal.

The LUT production apparatus 300 further comprises a class tapextraction circuit 303 for extracting as a class tap multiple items ofpixel data located in a periphery of a target position in the SD signaloutput from the SD signal production circuit 302 based on this SDsignal. This class tap extraction circuit 303 is configured similar tothe class tap extraction circuit 102 in the above-described apparatus100 for processing the image signal shown in FIG. 1.

The LUT production apparatus 300 further comprises a classcategorization circuit 304 for categorizing a class tap extracted by theclass tap extraction circuit 303 into any one of a plurality of classesbased on this class tap, to obtain a class code Ca indicative of a classof this class tap. This class categorization circuit 304 is configuredsimilar to the class categorization circuit 103 in the above-describedapparatus 100 for processing the image signal shown in FIG. 1.

The LUT production apparatus 300 further comprises a dynamic rangeprocessing circuit (DR processing circuit) 305. This DR processingcircuit 305 is configured similar to the DR processing circuit 104 inthe above-described apparatus 100 for processing the image signal shownin FIG. 1 and outputs “0” as area information AR if DR<Th while itoutputs “1” as the area information AR if DR<Th.

The LUT production apparatus 300 further comprises a coefficient memory306 for storing coefficient data Wi of each class, which is used in anestimate equation used in later-described predictive computation circuit309 and all-the-class predictive computation circuit 310. Thiscoefficient data Wi is information to convert an SD signal into an HDsignal. Coefficient data Wi stored in this coefficient memory 306 issupposed to be the same as coefficient data Wi to be stored in thecoefficient memory 106 in the apparatus 100 for processing the imagesignal shown in FIG. 1.

That is, the coefficient data Wi to be stored in this coefficient memory306 is obtained by learning between a student signal that corresponds tothe SD signal and a teacher signal that corresponds to the HD signal byuse of such a portion of the dynamic range DR as to satisfy DR≧Th. Thiscoefficient data Wi can be produced by using, for example, the apparatus200 for producing the coefficient data shown in FIG. 6.

The coefficient memory 306 is supplied with the class code Ca, as readaddress information, output from the above-described classcategorization circuit 304. The coefficient memory 306 outputscoefficient data Wi-a of a class indicated by the class code Ca.Further, from this coefficient memory 306, coefficient data Wi-q of eachof the classes is serially read by the later-described all-the-classpredictive computation circuit 310. It is to be noted that if there areN number of classes in all, q=1 to N. This coefficient memory 306constitutes first coefficient data generation means and secondcoefficient data generation means.

The LUT production apparatus 300 further comprises a delay circuit 307for time-adjusting the SD signal output from the SD signal productioncircuit 302 and a prediction tap extraction circuit 308 for extractingas a prediction tap multiple items of pixel data located in a peripheryof a target position in an SD signal output from this delay circuit 307based on this SD signal. This prediction tap extraction circuit 308 isconfigured similar to the prediction tap extraction circuit 107 in theabove-described apparatus 100 for processing the image signal shown inFIG. 1.

The LUT production apparatus 300 further comprises a predictivecomputation circuit 309. This predictive computation circuit 309 obtainspixel data y based on the above-described estimate equation (1) frompixel data xi as the prediction tap extracted by the prediction tapextraction circuit 308 and coefficient data Wi-a output from thecoefficient memory 306.

The LUT production apparatus 300 further comprises the all-the-classpredictive computation circuit 310. This all-the-class predictivecomputation circuit 310 serially reads coefficient data Wi-q of eachclass from the coefficient memory 306, to obtain pixel data y_(q) (q=1to N) based on the above-described estimate equation (1) from the pixeldata xi as a prediction tap extracted by the prediction tap extractioncircuit 308 and this coefficient data Wi-q.

The LUT production apparatus 300 further comprises a delay circuit 311for time-adjusting the HD signal input to the input terminal 301 and ateacher data extraction circuit 312 for extracting, as teacher data,pixel data that constitutes the HD signal output by this delay circuit311 and corresponds to a target position in the SD signal, based on thisHD signal.

The LUT production apparatus 300 further comprises a prediction errorcalculation circuit 313. This prediction error calculation circuit 313calculates an error E (p) of the pixel data y calculated by thepredictive computation circuit 309 with respect to the teacher data(true value) extracted by the teacher data extraction circuit 312. Inthis case, assuming the teacher data to be y₀, it can be obtained by E(p)=y₀−y. It is to be noted that p indicates a class number of a classindicated by a class code Ca obtained by the class categorizationcircuit 304.

The LUT production apparatus 300 further comprises an error additioncircuit 315 and an error memory 316 as error addition means anderror-sum accumulation means, respectively. This error addition circuit315 adds an error E (q) obtained by the prediction error calculationcircuit 313 with an error E (q) (q=1 to N) obtained by the all-the-classprediction error calculation circuit 314, to obtain an error sum (E(p)+E (q) (q=1 to N) of each class. Further, the error addition circuit315 adds a value that corresponds to a magnitude of each error sum ofeach class to an accumulated value of each output class at an inputclass that corresponds to a class code Ca obtained by the classcategorization circuit 304.

The LUT production apparatus 300 further comprises an error additioncircuit 315 and an error memory 316 as error addition means anderror-sum accumulation means, respectively. This pixel addition circuit315 adds an error E (p) obtained by the prediction error calculationcircuit 313 with an error E (q) (q=1 to N) obtained by the all-the-classprediction error calculation circuit 314, to obtain an error sum (E(p)+E (q)) (q=1 to N) of each class. Further, the error addition circuit315 adds a value that corresponds to a magnitude of each error sum ofeach class to an accumulated value of each output class at an inputclass that corresponds to a class code Ca obtained by the classcategorization circuit 304.

In this case, a value that corresponds to a magnitude of the error sum(E (p)+E (q)) is set to, for example, its squared value (E (p)+E (q))².It is to be noted that the value that corresponds to the magnitude ofthe error sum (E (p)+E (q)) may be its absolute value |(E (p)+E (q))|.

The error memory 316 stores an accumulated value of each output class ateach input class. The error addition circuit 315 reads from the errormemory 316 an accumulated value of each output class at an input class pthat corresponds to the class code Ca and, adds to the accumulated valueof each output class a value that corresponds to a magnitude of a newlyobtained error sum of each class, to provide a new accumulated value andthen writes back this new accumulated value to the error memory 316.

That is, the error memory 316 stores an accumulated value of each outputclass at each input class as shown in FIG. 8. To accumulated value ofeach of the output classes at an input class p that corresponds to aclass code Ca, values E1 to EN are added which correspond to magnitudesof the respective error sums of classes newly obtained.

It is to be noted that such components as the class categorizationcircuit 304, the coefficient memory 306, the prediction tap extractioncircuit 308, the predictive computation circuit 309, the all-the-classpredictive computation circuit 310, the teacher data extraction circuit312, the prediction error calculation circuit 313, the all-the-classprediction error calculation circuit 314, the error addition circuit315, and the error memory 316 perform processing only if areainformation AR output by the DR processing circuit 305 is “1”, that is,a dynamic range DR is less than a threshold value Th. Therefore, valuesE1 to EN that correspond to the magnitudes of the respective error sumsof classes newly obtained are added to accumulated values of outputclasses at an input class p that corresponds to the above-describedclass code Ca only if the dynamic range DR is less than the thresholdvalue Th.

The LUT production apparatus 300 further comprises an error minimumclass detection circuit 317 as table production means and a memory 318.This detection circuit 317 allocates output class in which anaccumulated-value is minimized to each input class based on anaccumulated value of each output class at each of the input classes,which is stored in the error memory 316. In such a manner, acorrespondence relationship between the input class and the output classis acquired.

For example, it is supposed that an accumulated value of each outputclass at an input class p is stored in the error memory 316 as shown inFIG. 9. In this case, for the input class p, an accumulated value “25”of an output class q is minimized. Therefore, the input class p isallocated the output class q.

Further, the detection circuit 317, based on the acquired correspondencerelationship between the input class and the output class, acquires acorrespondence relationship between a class code Ca that corresponds tothe input class and a class code Cb that corresponds to the output class(see FIG. 4) and stores it in the memory 318.

The following will describe operations of the LUT production apparatus300 shown in FIG. 7.

An HD signal is input to the input terminal 301 as a teacher signal. Onthis HD signal, horizontal and vertical thinning processing is performedby the SD signal production circuit 302, to produce an SD signal as astudent signal.

The SD signal obtained by the SD signal production circuit 302 issupplied to the class tap extraction circuit 303. This class tapextraction circuit 303, based on the SD signal, extracts as a class tapmultiple items of pixel data located in a periphery of a target positionin this SD signal (see FIG. 3A). This class tap is supplied to the classcategorization circuit 304 and the DR processing circuit 305.

The DR processing circuit 305 detects a dynamic range DR=MAX−MIN, whichis a difference between a maximum value MAX and a minimum value MIX ofitems of the pixel data contained in the class tap and, if this dynamicrange DR is not less than a threshold value Th, outputs “0” as areainformation AR while if the dynamic range DR is less than the thresholdvalue Th, it outputs “1” as the area information AR.

This area information AR is supplied to such components as the classcategorization circuit 304, the coefficient memory 306, the predictiontap extraction circuit 308, the predictive computation circuit 309, theall-the-class predictive computation circuit 310, the teacher dataextraction circuit 312, the prediction error calculation circuit 313,the all-the-class prediction error calculation circuit 314, the erroraddition circuit 315, and the error memory 316 as their operationcontrol signals. Processing by these components is performed only if thearea information AR is “1”, that is, the dynamic range DR is less thanthe threshold value Th.

Further, the class categorization circuit 304 performs data compressionprocessing such as ADRC processing on items of the pixel data containedin the class tap, to obtain a class code Ca indicative of a class ofthis class tap. This class code Ca is supplied to the coefficient memory306 as read address information and also to the error addition circuit315.

When the class code Ca is supplied as the read address information tothe coefficient memory 306, coefficient data Wi-a that corresponds to aclass indicated by the class code Ca is read from this coefficientmemory 306 and supplied to the predictive computation circuit 309.

The SD signal obtained by the SD signal production circuit 302 istime-adjusted by the delay circuit 307 and then supplied to theprediction tap extraction circuit 308. This prediction tap extractioncircuit 308, based on the SD signal, extracts as a prediction tapmultiple items of pixel data located in a periphery of a target positionin this SD signal (see FIG. 3B). Pixel data xi as this prediction tap issupplied to the predictive computation circuit 309 and the all-the-classpredictive computation circuit 310.

The predictive computation circuit 309 calculates pixel data y thatcorresponds to the target position in the SD signal based on theabove-described equation (1) from the pixel data xi and the coefficientdata Wi-a. This pixel data y is supplied to the prediction errorcalculation circuit 313. The all-the-class predictive computationcircuit 310 serially reads coefficient data Wi-q of each class from thecoefficient memory 306 and, from this coefficient data Wi-q and thepixel data xi, calculates items of pixel data y_(q) (q=1 to N) based onthe above-described estimate equation (1). This pixel data y_(q) issupplied to the all-the-class prediction error calculation circuit 314.

Further, the HD signal input to the input terminal 301 is time-adjustedby the delay circuit 311 and then supplied to the teacher dataextraction circuit 312. This teacher data extraction circuit 312extracts, as teacher data y₀, pixel data that constitutes the HD signaland corresponds to a target position in the SD signal. This teacher datay₀ is supplied to the prediction error calculation circuit 313 and theall-the-class prediction error calculation circuit 314.

The prediction error calculation circuit 313 calculates an error E(p)=y₀−y of pixel data y with respect to teacher data (true value) y₀.This error E (p) is supplied to the error addition circuit 315. It is tobe noted that p indicates a class number of a class indicated by theclass code Ca obtained by the class categorization circuit 304 asdescribed above. Further, the all-the-class prediction error calculationcircuit 314 calculates errors E (q)=y₀−y_(q) (q=1 to N) of pixel datay_(q) with respect to teacher data (true value) y₀. These errors E (q)(q=1 to N) are supplied to the error addition circuit 315.

The error addition circuit 315 adds the error E (p) with the respectiveerrors E (q) (q=1 to N) to obtain an error sum (E (p)+E (q)) (q=1 to N)of each class. Further, this error addition circuit 315 adds a valuethat corresponds to a magnitude of the obtained error sum of each class,for example, its squared value (E (p)+E (q))² to an accumulated value ofeach output class at an input class that corresponds to the class codeCa obtained by the class categorization circuit 304.

In this case, an accumulated value of each output class at each inputclass is stored in the error memory 316, from which an accumulated valueof each output class at an input class p that corresponds to the classcode Ca is read, and to the accumulated value of each output class, avalue is added which corresponds to a magnitude of an error sum of eachclass newly obtained, thereby to provide a new accumulated value, whichis then written back to the error memory 316 (See FIG. 8).

In such a manner, processing to add a value that corresponds to amagnitude of an error sum of each value newly obtained to an accumulatedvalue of each output class at an input class p that corresponds to aclass code Ca is serially performed corresponding to each targetposition in the SD signal. It is to be noted that the processing by suchcomponents as the class categorization circuit 304, the coefficientmemory 306, the prediction tap extraction circuit 308, the predictivecomputation circuit 309, the all-the-class predictive computationcircuit 310, the teacher data extraction circuit 312, the predictionerror calculation circuit 313, the all-the-class prediction errorcalculation circuit 314, the error addition circuit 315, and the errormemory 316 is performed only if the area information AR is “1”, that is,the dynamic range DR is less than a threshold value Th as describedabove, so that accumulation processing by the above-described erroraddition circuit 315 is performed only if the dynamic range DR is lessthan the threshold value Th.

The error minimum class detection circuit 317 allocates to each inputclass an output class in which an accumulated value is minimized basedon an accumulated value of each output class at each of the inputclasses stored in the error memory 316 (see FIG. 9), thereby obtaining acorrespondence relationship between the input class and the outputclass. This detection circuit 317 further acquires a correspondencerelationship (see FIG. 4) between a class code Ca that corresponds to aninput class and a class code Cb that corresponds to an output classbased on an acquired input-class vs. output class correspondencerelationship. This correspondence relationship is stored in the memory318.

In such a manner, in the LUT production apparatus 300 shown in FIG. 7,it is possible to produce a correspondence relationship between a classcode Ca and a class code Cb which are stored in the LUT105 in theapparatus 100 for processing the image signal of FIG. 1. In this case,as described above, to each input class, an output class in which anaccumulated value that corresponds to a magnitude of an error sum isminimized is allocated, so that it is possible to convert the class codeCa into the class code Cb in such a manner that an addition mean valueof pixel data calculated by using coefficient data Wi that correspondsto the class code Ca and pixel data calculated by using coefficient dataWi that corresponds to the class code Cb may most approach a true valueof pixel data that constitutes the HD signal.

It is to be noted that the processing in the above-described apparatus100 for processing the image signal of FIG. 1 can be performed bysoftware by using an image signal processing apparatus (computer) 500like one as shown in FIG. 10, for example.

First, the image signal processing apparatus 500 shown in FIG. 10 willbe described. This image signal processing apparatus 500 comprises aCPU501 for controlling operations of the apparatus as a whole, a readonly memory (ROM) 502 for storing a control program of this CPU501,coefficient data Wi, a correspondence relationship (lookup table)between a class code Ca and a class code Cb, etc., and a random accessmemory (RAM) 503 that constitutes a working space for the CPU501. TheseCPU501, ROM502, and RAM503 are each connected to a bus 504.

The image signal processing apparatus 500 further comprises a hard discdrive (HDD) 505 as an external storage device and a drive 506 forhandling a removable recording medium such as a flexible disc, a compactdisc read only memory (CD-ROM), a magneto-optical (MO) disc, a digitalversatile disc (DVD), a magnetic disk, and a semiconductor memory. Thesedrives 505 and 506 are each connected to the bus 504.

The image signal processing apparatus 500 further comprises acommunication section 508 for connecting to a communication network 507such as the Internet in a wired or wireless manner. This communicationsection 508 is connected to the bus 504 via an interface 509.

The image signal processing apparatus 500 further comprises a userinterface section. This user interface section comprises aremote-control signal reception circuit 511 for receiving aremote-control signal RM from a remote-controlled transmitter 510 and adisplay 513 constituted of a cathode-ray tube (CRT), a liquid crystaldisplay (LCD), etc. The reception circuit 511 is connected to the bus504 via an interface 512 and the display 513 is similarly connected tothe bus 504 via an interface 514.

The image signal processing apparatus 500 further comprises an inputterminal 515 for inputting an SD signal and an output terminal 517 foroutputting an HD signal. The input terminal 515 is connected to the bus504 via an interface 516 and the output terminal 517 is similarlyconnected to the bus 504 via an interface 518.

It is to be noted that instead of storing the control program etc. inthe ROM502 as described above beforehand, it may be downloaded via thecommunication section 508 from the communication network 507 such as theInternet for example so that it can be stored in the hard disc drive 505or the RAM 503 and used. Further, the control program etc. may beprovided in a removable recording medium.

Further, instead of inputting via the input terminal 515 an SD signal tobe processed, it may be supplied in a removable recording medium ordownloaded via the communication section 508 from the communicationnetwork 507 such as the Internet. Further, instead of or concurrentlywith outputting the post-processing HD signal to the output terminal517, it may be supplied to the display 513 to display an image or storedin the hard disc drive 505 to be sent via the communication section 508to the communication network 507 such as the Internet.

The following will describe a processing procedure for obtaining an HDsignal from an SD signal in the image signal processing apparatus 500shown in FIG. 10, with reference to a flowchart of FIG. 11.

First, the process starts at step ST11 and, at step ST12, inputs an SDsignal of one frame or one field from, for example, the input terminal515 into the apparatus. The SD signal thus input is temporarily storedin the RAM 503.

At step ST13, the process decides whether all of the frames or fields inthe SD signal have been processed. If they have been processed, theprocess ends the processing at step ST14. Otherwise, if they have notbeen processed, the process goes to step ST15.

At step ST15, the process extracts as a class tap multiple items ofpixel data located in a periphery of a target position in the SD signalinput at step ST12 based on this SD signal (see FIG. 3A). Then, at stepST16, the process extracts as a prediction tap multiple items of pixeldata located at a periphery of a target position in the SD signal inputat step ST12 based on this SD signal (see FIG. 3B).

Next, at step ST17, based on the class tap extracted at step ST15, theprocess categorizes this class tap into any one of a plurality ofclasses to thereby obtain a class code Ca. At step ST18, the processacquires, from the ROM 502, coefficient data Wi-a, which is used in anestimate equation, corresponding to the class code Ca acquired at stepST17.

At step ST19, the process uses the pixel data xi extracted as aprediction tap at step ST16 and the coefficient data Wi-a acquired atstep ST18, to produce four items of pixel data y_(1-n) to y_(4-n) thatcorrespond to a target position in the SD signal based on the estimateequation (see Equation (1)).

Next, at step ST20, based on the class tap extracted at step ST15, theprocess calculates a dynamic range DR=MAX−MIN, which is a differencebetween a maximum value MAX and a minimum value MIN of items of thepixel data contained in that class tap. At step ST21, the processdecides whether DR<Th, that is, a dynamic range DR is less than athreshold value Th.

If not DR<Th, that is, if the dynamic range DR is not less than thethreshold value Th, the process decides that the items of pixel datay_(1-n) to y_(4-n) calculated at step ST19 are items of pixel datay_(1-a) to y_(4-a) that constitute the HD signal and goes to step ST26.If DR<Th, on the other hand, the process goes to step ST22.

AT this step ST22, the process converts the class code Ca acquired atstep ST17 into a class code Cb according to a correspondencerelationship between the class codes Ca and Cb stored in the ROM 502. Atstep ST23, the process acquires, from the ROM 502, coefficient dataWi-b, which is used in the estimate equation, corresponding to the classcode Cb obtained by conversion at step ST22.

At step ST24, the process uses the pixel data xi as a prediction tapextracted at step ST16 and the coefficient data Wi-b acquired at stepST23, to produce four items of pixel data y_(1-n) to y_(4-n) thatcorrespond to a target position in the SD signal based on the estimateequation (see Equation (1)).

Next, at step ST25, the process obtains an addition mean value(y_(1-a)+y_(1-b))/2 through (y_(4-a)+y_(4-b))/2 of four items of pixeldata y_(1-a)+y_(4-a) calculated at step ST19 and four items of pixeldata y_(1-b)+y_(4-b) calculated at step ST24 and decides this as itemsof pixel data y₁ to y₄ that constitute the HD signal and then goes tostep ST26.

At step ST26, the process decides whether the processing to obtain thepixel data of the HD signal has ended over all of regions of the pixeldata of the one frame or one field of the SD signal input at step ST12.If it has ended, the process returns to step ST12 where processing toinput the SD signal of next one frame or one field is shifted. If it hasnot ended, the process returns to step ST15 where processing on the nexttarget position in the SD signal is shifted.

In such a manner, by performing the processing along the flowchart shownin FIG. 11, it is possible to obtain an HD signal from an SD signal bythe same method as that in the apparatus 100 for processing the imagesignal shown in FIG. 1. The HD signal thus obtained is output to theoutput terminal 517 or supplied to the display 513 so that an image dueto it may be displayed or supplied to the hard disk drive 505 so that itmay be recorded.

Although a processing apparatus for it is not shown, the processing inthe apparatus 200 for producing the coefficient data shown in FIG. 6 canalso be performed by software.

The following will describe a processing procedure for producingcoefficient data Wi with reference to the flowchart of FIG. 12.

First, the process starts at step ST31 and, at step ST32, inputs an HDsignal of one frame or one field as a teacher signal. At step ST33, theprocess decides whether all of the frames or fields in the HD signalhave been processed. If they have not been processed, at step ST34 theprocess produces an SD signal as a student signal from the HD signalinput at step ST32.

Next, at step ST35, based on the SD signal produced at step ST34, theprocess extracts as a class tap multiple items of pixel data located ina periphery of a target position in this SD signal (see FIG. 3A). Atstep ST36, based on the class tap extracted at step ST35, the processcalculates a dynamic range DR=MAX−MIN, which is a difference between amaximum value MAX and a minimum value MIN of items of the pixel datacontained in the class tap. At step ST37, the process decides whetherDR<Th, that is, whether the dynamic range DR is less than the thresholdvale Th.

If DR<Th, that is, if the dynamic range DR is less than the thresholdvalue Th, the process directly goes to step ST41. On the other hand, ifnot DR<Th, that is, if the dynamic range DR is not less than thethreshold value Th, the process goes to step ST38. At this step ST38,based on the class tap extracted at step ST35, the process categorizesthis class tap as any one of a plurality of classes to thereby acquire aclass code Ca.

Next, at step ST39, based on the SD signal produced at step ST34, theprocess extracts as a prediction tap multiple items of pixel datalocated in a periphery of a target position in this SD signal (see FIG.3B). At step ST40, the process uses the class code Ca acquired at stepST38, pixel data xi of the prediction tap extracted at step ST39, anditems of pixel data (teacher data) y₁ to y₄, that correspond to thetarget position in the SD signal, of the HD signal input at step ST32 tothereby perform supplementation required to obtain a normal equationgiven in Equation (8) for each class (see Equations (6) and (7)).

Next, at step ST41, the process decides whether learning processing hasended over all of regions of the pixel data of the one frame or onefield in the HD signal input at step ST32. If the learning processinghas ended, the process returns to step ST32 where the HD signal of thenext one frame or one field is input and then the same processing asabove repeats. On the other hand, if the learning processing has notended, the process returns to step ST35 where processing on the nexttarget position in the SD signal is shifted.

If the processing has ended at the above-described step ST33, at stepST42 the process solves the normal equation produced at step ST40 toobtain coefficient data Wi of each class and saves it in the coefficientmemory at step ST43, to end the processing at step ST44.

In such a manner, by performing the processing along the flowchart shownin FIG. 12, it is possible to produce the coefficient data Wi by thesame method as that in the apparatus 200 for producing the coefficientdata shown in FIG. 6.

Although a processing apparatus for it is not shown, the processing inthe LUT production apparatus 300 shown in FIG. 7 can also be performedby software. The following will describe a processing procedure forproducing a correspondence relationship between a class code Ca and aclass code Cb with reference to a flowchart of FIG. 13.

First, the process starts at step ST51 and, at step ST52, inputs an HDsignal of one frame or one field as a teacher signal. At step ST53, theprocess decides whether all of the frames or fields in the HD signalhave been processed. If they have not processed, at step ST54, theprocess produces an SD signal as a student signal from the HD signalinput at step ST52.

Next, at step ST55, based on the SD signal produced at step ST54, theprocess extracts as a class tap multiple items of pixel data located ina periphery of a target position in this SD signal (see FIG. 3A). Atstep ST56, based on the class tap extracted at step ST55, the processcalculates a dynamic range DR=MAX−MIN, which is a difference between amaximum value MAX and a minimum value MIN of items of the pixel datacontained in the class tap. At step ST57, the process decides whetherDR<Th, that is, whether the dynamic range DR is less than the thresholdvale Th.

If not DR<Th, that is, if the dynamic range DR is not less than athreshold value Th, the process directly goes to step ST66. On the otherhand, if DR<Th, the process goes to step ST58. At this step ST58, basedon the class tap extracted at step ST55, the process categorizes thisclass tap as any one of a plurality of classes to thereby acquire aclass code Ca. A class number that this class code Ca indicates issupposed to be p.

Next, at step ST59, based on the SD signal produced at step ST54, theprocess extracts as a prediction tap multiple items of pixel datalocated in a periphery of a target position in this SD signal (see FIG.3B). At step ST60, the process uses pixel data xi of the prediction tapextracted at step ST59 and coefficient data Wi-a that corresponds to theclass code Ca acquired at step ST58, to obtain pixel data y thatcorresponds to the target position in the SD signal based on theabove-described Equation (1).

Furthermore, at this step ST60, the process extracts, as teacher datay₀, pixel data that corresponds to the target position in the SD signalbased on the HD signal input at step ST52, to calculate an error E(p)=y₀−y of the pixel data y with respect to its teacher data (truevalue) y₀.

Next, at step ST61, the process sets q=1. At step ST62, the process usespixel data xi of the prediction tap extracted at step ST59 andcoefficient data Wi-q that corresponds to a class number q, to obtainpixel data y_(q) that corresponds to the target position in the SDsignal based on the above-described Equation (1). Furthermore, at thisstep ST62, the process extracts, as teacher data y₀, pixel data thatcorresponds to the target position in the SD signal based on the HDsignal input at step ST52, to calculate an error E(q)=y₀−y_(q) of thepixel data y_(q) with respect to its teacher data (true value) y₀.

Next, at step ST63, the process adds E (p) calculated at step ST60 andan error E (q) calculated at step ST62 to obtain an error sum (E (p)+E(q)). Furthermore, at step ST63, the process adds a value thatcorresponds to a magnitude of this error sum, for example, a squared-sum(E (p)+E (q))² to an accumulated value of an output class q at an inputclass p.

Next, at step ST64, the process decides whether q<N. If not q<N, thatis, if the processing has ended over all of the output classes, theprocess goes to step ST66. If q<N and the processing has not ended overall of the output classes, on the other hand, the process increases q byone at step ST65 and returns to step ST62 where an error sum for thenext output class is acquired and processing to add it to the relevantaccumulated value having the corresponding value is shifted.

At step ST66, the process decides whether the processing has ended overall of regions of pixel data of the one frame or one field in the HDsignal input at step ST52. If the processing has ended, the processreturns to step ST52 where the HD signal of the next one frame or onefield is input and then the same processing as above repeats. If theprocessing has not ended, on the other hand, the process returns to stepST55 where the processing on the next target position in the SD signalis shifted.

If the processing has ended at the above-described step ST53, at stepST67, the process allocates an output class in which an accumulatedvalue is minimized for each input class based on an accumulated value ofeach output class at each of the input classes, to acquire acorrespondence relationship between the input class and the outputclass. Furthermore, at this step ST67, the process acquires acorrespondence relationship (see FIG. 4) between a class code Ca thatcorresponds to an input class and a class code Cb that corresponds to anoutput class based on the acquired correspondence relationship betweenthe input class and the output class and saves it in the memory.

After the processing at step ST67, the process ends at step ST68.

By thus performing the processing along the flowchart shown in FIG. 13,it is possible to produce a correspondence relationship between a classcode Ca and a class code Cb by the same method as that in the LUTproduction apparatus 300 shown in FIG. 7.

The following will describe another embodiment of the present invention.FIG. 14 shows a configuration of an apparatus 100A for processing animage signal according to another embodiment. This apparatus 100A forprocessing the image signal converts an SD signal as a first informationsignal into an HD signal as a second information signal. In this FIG.14, the same components as those in FIG. 1 are indicated by the samesymbols and their detailed explanation will be omitted.

This apparatus 100A for processing the image signal comprises acoefficient memory 106A. This coefficient memory 106A stores coefficientdata Wis for each class and coefficient data Wic common to the classesthat are used in an estimate equation which is used in a later-describedpredictive computation circuit 108A. Each of the items of coefficientdata Wis and Wic is information used to convert an SD signal into an HDsignal.

As described above, when converting the SD signal into the HD signal, itis necessary to obtain four pixels (y1 to y4) of the HD signalcorresponding to one pixel (x₀) of the SD signal (see FIG. 2). In thiscase, the four pixels of the HD signal have different shifts in phasewith respect to the one pixel of the corresponding SD signal. Therefore,the items of coefficient data Wis and Wic are stored in the coefficientmemory 106A for each combination of classes and output pixel positions(positions of y₁ to y₄).

Class-specific coefficient data Wis is the same as the class-specificcoefficient data Wi stored in the coefficient memory 106 in theapparatus 100 for processing the image signal shown in FIG. 1. That is,this coefficient data Wi is obtained through learning between a studentsignal (first learning signal) that corresponds to the SD signal and ateacher signal (second learning signal) that corresponds to the HDsignal by use of such a portion of a dynamic range DR as to satisfy arelationship of DR≧Th.

On the other hand, coefficient data Wic common to classes is based on aresult of learning, without class categorization, between a studentsignal (first learning signal) that corresponds to the SD signal and ateacher signal (second learning signal) that corresponds to the HDsignal.

For example, the class-specific coefficient data Wis can be produced inthe above-described apparatus 200 for producing the coefficient datashown in FIG. 6. The coefficient data Wic common to the classes can beproduced by a configuration of this apparatus 200 for producing thecoefficient data excluding the components of the class tap extractioncircuit 203, the class categorization circuit 204, and the DR processingcircuit 205. In this case, in the learning pair storage section 210,multiple items of learning pair data are stored irrespective of classcategorization. This computation circuit 211 produces a normal equationwhich obtains the coefficient data Wic common to classes using themultiple items of learning pair data, so that by solving this equation,this class-common coefficient data Wic can be obtained.

Referring back to FIG. 14, the coefficient memory 106A is supplied witha class code Ca obtained by the above-described class categorizationcircuit 103 and area information AR obtained by the DR processingcircuit 104. If the area information AR is “0” and a dynamic range DR isnot less than a threshold value Th, the coefficient memory 106A outputs,as coefficient data Wi, coefficient data Wis of a class indicated by theclass code Ca among items of the class-specific coefficient data Wiswhile if the area information AR is “1” and the dynamic range DR is lessthan the threshold value Th, it outputs class-common coefficient dataWic as the coefficient data Wi.

The apparatus 100A for processing the image signal further comprises thepredictive computation circuit 108A. This predictive computation circuit108A obtains items of pixel data y₁ to y₄ that constitute the HD signaland correspond to a target position in the SD signal based on anestimate equation of the above-described Equation (1) from pixel data xias a prediction tap extracted by the prediction tap extraction circuit107 and the coefficient data Wi output from the coefficient memory 106A.

The other components of the apparatus 100A for processing the imagesignal are configured the same manner as those of the apparatus 100 forprocessing the image signal shown in FIG. 1.

The following will describe operations of the apparatus 100A forprocessing the image signal shown in FIG. 14.

An SD signal input to the input terminal 101 is supplied to the classtap extraction circuit1 102. Based on the SD signal, this class tapextraction circuit 102 extracts as a class tap multiple items of pixeldata located in a periphery of a target position in this SD signal (seeFIG. 3A). This class tap is supplied to the class categorization circuit103 and the DR processing circuit 104.

The class categorization circuit 103 performs data compressionprocessing such as ADRC processing on items of pixel data contained inthe class tap, to obtain a class code Ca indicative of a class of thisclass tap. This class code Ca is supplied as read address information tothe coefficient memory 106A.

Further, the DR processing circuit 104 detects a dynamic rangeDR=MAX−MIN, which is a difference between a maximum value MAX and aminimum value MIN of items of the pixel data contained in the class tapand if this dynamic range DR is not less than the threshold value Th, itoutputs “0” as the area information AR and if this dynamic range DR isless than the threshold value Th, on the other hand, it outputs “1” asthe area information AR. This area information AR is supplied as readaddress information to the coefficient memory 106A.

If the area information AR is “0” and the dynamic range DR is not lessthan the threshold value Th, coefficient data Wis of a class indicatedby the class code Ca among items of the class-specific coefficient dataWis is output from the coefficient memory 106A as coefficient data Wi.If the area information AR is “1” and the dynamic range DR is less thanthe threshold value Th, the class-common coefficient data Wic is outputfrom the coefficient memory 106A as coefficient data Wi. The coefficientdata Wi thus output from the coefficient memory 106A is supplied to thepredictive computation circuit 108A.

Further, the SD signal input to the input terminal 101 is supplied tothe prediction tap extraction circuit 107. Based on the SD signal, thisprediction tap extraction circuit 107 extracts as a prediction tapmultiple items of pixel data located in a periphery of a target positionin this SD signal (see FIG. 3B). Pixel data xi as this prediction tap issupplied to the predictive computation circuit 108A.

The predictive computation circuit 108A calculates four items of pixeldata y₁ to y₄ that constitute the HD signal and correspond to a targetposition in the SD signal based on Equation (1) using the pixel data xiand the coefficient data Wi. Thus, the four items of pixel data y₁ to y₄that constitute the HD signal, correspond to each target position in theSD signal and are calculated serially by the predictive computationcircuit 108A are supplied to the post-processing circuit 110.

This post-processing circuit 110 linear-serializes the four items ofpixel data y₁ to y₄ that constitute the HD signal, correspond to thetarget position in the SD signal, and are serially supplied from thepredictive computation circuit 108A, thereby obtaining the HD signal.This HD signal is output to the output terminal 111.

In the above-described apparatus 100A for processing the image signal,if the area information AR is “0”, that is, the dynamic range DR is notless than the threshold value Th, the items of pixel data y₁ to y₄ thatconstitute the HD signal are obtained by using the coefficient data Wisof a class indicated by a class code Ca. In this case, as describedabove, the coefficient data Wis is obtained through learning between astudent signal (first learning signal) that corresponds to the SD signaland a teacher signal (second learning signal) that corresponds to the HDsignal by use of such a portion of the dynamic range DR as to be notless than the threshold value Th, so that the items of pixel data y₁ toy₄ that constitute the HD signal can be obtained accurately.

Further, in the above-described apparatus 100A for processing the imagesignal, if the area information AR is “1”, that is, the dynamic range DRis less than the threshold value Th, the items of pixel data y₁ to y₄that constitute the HD signal are obtained by using the class-commoncoefficient data Wic. In this case, the coefficient data Wic is based ona result of learning, without class categorization, between a studentsignal (first learning signal) that corresponds to the SD signal and ateacher signal (second learning signal) that corresponds to the HDsignal. Therefore, the coefficient data Wic is an average value of theitems of coefficient data of the classes, so that errors of the items ofthe pixel data y₁ to y₄ that constitute the HD signal calculated byusing this coefficient data Wic with respect to their true values aredistributed around error 0.

Therefore, by the above-described apparatus 100A for processing theimage signal, it is possible to well obtain the items of pixel data y₁to y₄ that constitute an HD signal no matter whether the dynamic rangeDR is large or small as in the case of the apparatus 100 for processingthe image signal shown in FIG. 1. Further, by this apparatus 100A forprocessing the image signal, the lookup table 105 required in theapparatus 100 for processing the image signal shown in FIG. 1 can beomitted, thereby saving on a memory capacity of the system as a whole.

It is to be noted that the above-described processing in the apparatus100A for processing the image signal shown in FIG. 14 can be performedby software in the image signal processing apparatus (computer) 500shown in FIG. 10 for example.

The following will describe a processing procedure for obtaining an HDsignal from an SD signal in the image signal processing apparatus 500shown in FIG. 9, with reference to a flowchart of FIG. 15.

First, the process starts at step ST71 and, at step ST72, it inputs anSD signal of one frame or one field from, for example, the inputterminal 515 into the apparatus. The SD signal thus input is temporarilystored in the RAM 503.

At step ST73, the process decides whether all of the frames or fields ofthe SD signal are processed. If they are processed, the process ends theprocessing at step ST74. If they are not processed, on the other hand,the process goes to step ST75.

At step ST75, the process extracts as a class tap multiple items ofpixel data located in a periphery of a target position in the SD signalinput at step ST72 based on this SD signal (see FIG. 3A). Then, at stepST76, the process extracts as a prediction tap multiple items of pixeldata located in a periphery of a target position in the SD signal inputat step ST72 based on this SD signal (see FIG. 3B).

Next, at step ST77, based on the class tap extracted at step ST75, theprocess categorizes this class tap as any one of a plurality of classes,to obtain a class code Ca. At step ST78, based on the class tapextracted at step ST75, the process calculates a dynamic rangeDR=MAX−MIN, which is a difference between a maximum value MAX and aminimum value MIN of items of the pixel data contained in that classtap.

Next, at step ST79, the process acquires coefficient data Wi based onthe class code Ca acquired at step ST77 and the dynamic range DRcalculated at step ST78. In this case, if the dynamic range DR is notless than a threshold value Th, from the ROM 502, the process acquires,as coefficient data Wi, coefficient data Wis of a class indicated by theclass code Ca among items of the class-specific coefficient data Wis. Ifthe dynamic range DR is less than the threshold value Th, on the otherhand, the process acquires class-common coefficient data Wic ascoefficient data Wi from the ROM502.

Next, at step ST80, the process produces four items of pixel data y₁ toy₄ that correspond to a target position in the SD signal based on theestimate equation (see Equation (1)) by using the pixel data xiextracted as the prediction tap at step ST76 and the coefficient data Wiacquired at step ST79.

Next, at step ST81, the process decides whether the processing to obtainthe pixel data of the HD signal has ended over all of regions of thepixel data of the one frame or one field in the SD signal input at stepST72. If it has ended, the process returns to step ST72 where processingto input the next one frame or one field of the SD signal is shifted. Ifit has not ended, on the other hand, the process returns to step ST75where processing on the next target position in the SD signal isshifted.

In such a manner, by performing the processing along the flowchart shownin FIG. 15, it is possible to obtain the HD signal from the SD signal bythe same method as that in the apparatus 100A for processing the imagesignal shown in FIG. 14.

The following will describe a further embodiment of the presentinvention. FIG. 16 shows a configuration of an apparatus 100B forprocessing an image signal according to this further embodiment. Incontract to the apparatus 100 for processing the image signal shown inFIG. 1 in which coefficient data Wi of each class is stored in thecoefficient memory 106 beforehand, in the apparatus 100B for processingthe image signal shown in FIG. 16, ROM stores coefficient seed data,which is coefficient data in a production equation for producing thecoefficient data Wi of each class, whereby the coefficient data Wi isproduced by using this coefficient seed data. In this FIG. 16, the samecomponents as those in FIG. 1 are indicated by the same symbols andtheir detailed explanation will be omitted.

This apparatus 100B for processing the image signal comprises an ROM112. In this ROM 112, coefficient seed data of each class is storedbeforehand. This coefficient seed data is coefficient data for aproduction equation that produces coefficient data Wi to be stored inthe coefficient memory 106.

As described above, the predictive computation circuits 108 a and 108 bcompute pixel items of pixel data y_(1-a) to y_(4-a) and y_(1-b)+y_(4-b)by an estimate equation of Equation (1) from the pixel data xi as aprediction tap and items of the coefficient data Wi-a and Wi-b read fromthe coefficient memory 106.

Items of coefficient data Wi (i=1 to N) used in the estimate equationand to be stored in the coefficient memory 106 are produced by aproduction equation including parameters r and z as shown in Equation(9). In this equation, r is a parameter which determines resolution andz is a parameter which determines a degree of noise rejection. In theROM 112, items of coefficient seed data w_(i0) to w_(i9) (i=1 to N),which are items of coefficient data for this production equation, arestored for each combination of classes and output pixel positions(positions of y₁ to y₄, see FIG. 2). How to produce this coefficientseed data will be described later.

$\begin{matrix}{{Wi} = {w_{i\; 0} + {w_{i\; 1}r} + {w_{i\; 2}z} + {w_{i\; 3}r^{2}} + {w_{i\; 4}{rz}} + {w_{i\; 5}z^{2}} + {w_{i\; 6}r^{3}} + {w_{i\; 7}r^{2}z} + {w_{i\; 8}{rz}^{2}} + {w_{i\; 9}z^{3}}}} & {{Equation}\mspace{14mu}(9)}\end{matrix}$

The apparatus 100B for processing the image signal further comprises acoefficient production circuit 113 for producing coefficient data Wi,which is used in an estimate equation and corresponds to the values ofthe parameters r and z for each combination of classes and output pixelpositions based on Equation (9) using coefficient seed data of eachclass and values of the parameters r and z. Into this coefficientproduction circuit 113, items of the coefficient seed data wi0 to w_(i9)are loaded from the ROM 112. Further, the parameters r and z are alsosupplied to this coefficient production circuit 113.

Coefficient data Wi of each class produced by this coefficientproduction circuit 109 is stored in the above-described coefficientmemory 106. The production of the coefficient data Wi in thiscoefficient production circuit 113 is performed, for example, for eachvertical blanking period. Accordingly, even if the values of theparameters r and z are changed by a user operation, it is possible toreadily change coefficient data Wi of each class stored in thecoefficient memory 106 to a value that corresponds to the values of suchthe parameters r and z, thereby permitting the user to smoothly adjustthe resolution and the degree of noise rejection thereof.

It is to be noted that items of the coefficient seed data w_(i0) tow_(i9) stored in the ROM 112 have been obtained through learning betweena student signal (first learning signal) that corresponds to the SDsignal and a teacher signal (second learning signal) that corresponds tothe HD signal by use of such a portion of a dynamic range DR as tosatisfy a relationship of DR≧Th as in the case of the coefficient dataWi which is stored beforehand in the coefficient memory 106 in theabove-described apparatus 100 for processing the image signal shown inFIG. 1.

Therefore, similar to the coefficient data Wi which is stored beforehandin the coefficient memory 106 in the apparatus 100 for processing theimage signal shown in FIG. 1, the coefficient data Wi to be stored inthe coefficient memory 106 in this apparatus 100B for processing theimage signal shown in FIG. 16 is also based on a result of learningbetween a student signal (first learning signal) that corresponds to theSD signal and a teacher signal (second learning signal) that correspondsto the HD signal by use of such a portion of the dynamic range DR as tosatisfy the relationship of DR≧Th.

The other components of the apparatus 100B for processing the imagesignal are configured and operate the same way as those of the apparatus100 for processing the image signal shown in FIG. 1.

The processing by the apparatus 100B for processing the image signalshown in FIG. 16 can also be realized by software. A processingprocedure in this case is roughly the same as that for the image signalprocessing shown in FIG. 11. However, at steps ST18 and ST23, items ofcoefficient seed data w_(i0) to w_(i9) that correspond to theirrespective class codes Ca are used to produce items of coefficient dataWi-a and Wi-b that correspond to parameters r and z that are set by theuser.

The following will describe how to produce items of the coefficient seeddata w_(i0) to w_(i9) (i=1 to N) of each class, which are stored in theROM 112. The items of coefficient seed data w_(i0) to w_(i9) areproduced by learning. How to learn will be described below.

For ease of explanation, tj (j=0 to 9) is defined as given in Equation(10).t₀1, t₁=r, t₂=z, t₃=r², t₄=rz, t₅=z², t₆=r³, t₇=r²z, t₈=rz²,t₉=z³  Equation (10)

By using this Equation (10), Equation (9) is rewritten as Equation (11).

$\begin{matrix}{{Wi} = {\sum\limits_{j = 0}^{9}{W_{ij}t_{j}}}} & {{Equation}\mspace{14mu}(11)}\end{matrix}$

Finally, an undetermined coefficient w_(ij) is obtained by learning.That is, by using multiple items of SD pixel data and HD pixel data foreach combination of classes and output pixel positions, a coefficientvalue that minimizes a square error is determined. This solution employsso-called the least-squares method. Assuming the number of times oflearning to be m, a remainder of the k'th learning data (1≦k≦m) to bee_(k), and a total sum of the square errors to be E, E can be given byEquation (12) based on Equations (1) and (9). In it, x_(ik) indicatesthe k'th item of pixel data at the i'th prediction tap position of an SDimage and y_(k) indicates pixel data of the corresponding k'th HD imagecorresponding thereto.

$\begin{matrix}\begin{matrix}{E = {\sum\limits_{k = 1}^{m}e_{k^{2}}}} \\{= {\sum\limits_{k = 1}^{m}\left\lbrack {{yk} - \left( {W_{1x\; 1k} + W_{2\; x\; 2\; k} + \ldots + {W_{n}x_{nk}}} \right)} \right\rbrack^{2}}} \\{= {\sum\limits_{k = 1}^{m}\left\{ {{yk} - \left\lbrack {{\left( {{t_{0}w_{10}} + {t_{1}w_{11}} + \ldots + {t_{9}w_{19}}} \right)x_{1k}} + \ldots + \left( {{t_{0}w_{n\; 0}} + {t_{1}w_{n\; 1}} + \ldots + {t_{9}w_{n\; 9}}} \right)} \right\rbrack} \right\}^{2}}} \\{= {\sum\limits_{k = 1}^{m}\left\{ {{yk} - \left\lbrack {{\left( {w_{10} + {w_{11}r} + \ldots + {w_{19}z^{3}}} \right)x_{1k}} + \ldots + {\left( {w_{n\; 0} + {w_{n\; 1}r} + \ldots + {w_{n\; 9}z^{3}}} \right)x_{nk}}} \right\rbrack} \right\}^{2}}}\end{matrix} & {{Equation}\mspace{14mu}(12)}\end{matrix}$

By the solution based on the least-squares method, such w_(ij) isobtained that a partial differentiation by use of w_(ij) in Equation(12) may be 0. This is indicated by Equation (13).

$\begin{matrix}{\frac{\partial E}{\partial{wij}} = {{\sum\limits_{k = 1}^{m}{2\left( \frac{\partial{ek}}{\partial{wij}} \right){ek}}} = {{- {\sum\limits_{k = 1}^{m}{2t_{j}x_{ik}e_{k}}}} = 0}}} & {{Equation}\mspace{14mu}(13)}\end{matrix}$

Similarly, by defining X_(ipjq) and Y_(ip) as in Equations (14) and(15), Equation (13) can be rewritten as Equation (16) by using a matrix.

$\begin{matrix}{X_{ipjq} = {\sum\limits_{k = 1}^{m}{x_{ik}t_{p}x_{jk}t_{q}}}} & {{Equation}\mspace{14mu}(14)}\end{matrix}$

$\begin{matrix}{Y_{ip} = {\sum\limits_{k = 1}^{m}{x_{ik}t_{p}x_{yk}}}} & {{Equation}\mspace{14mu}(15)}\end{matrix}$

$\begin{matrix}{\begin{bmatrix}x_{1010} & x_{1011} & x_{1012} & \cdots & x_{1019} & x_{1020} & \cdots & x_{10n\; 9} \\x_{1110} & x_{1111} & x_{1112} & \cdots & x_{1119} & x_{1120} & \cdots & x_{11n\; 9} \\x_{1210} & x_{1211} & x_{1211} & \cdots & x_{1219} & x_{1220} & \cdots & x_{12n\; 9} \\\vdots & \vdots & \vdots & ⋰ & \vdots & \vdots & ⋰ & \vdots \\x_{1910} & x_{1911} & x_{1912} & \cdots & x_{1919} & x_{1920} & \cdots & x_{19n\; 9} \\x_{2020} & x_{2011} & x_{2012} & \cdots & x_{2019} & x_{2020} & \cdots & x_{20n\; 9} \\\vdots & \vdots & \vdots & ⋰ & \vdots & \vdots & ⋰ & \vdots \\x_{n\; 910} & x_{n\; 911} & x_{n\; 912} & \cdots & x_{n\; 919} & x_{n\; 920} & \cdots & x_{n\; 9n\; 9}\end{bmatrix}{\quad{\begin{bmatrix}w_{10} \\w_{11} \\w_{12} \\\vdots \\w_{19} \\w_{20} \\\vdots \\w_{n\; 9}\end{bmatrix} = \begin{bmatrix}y_{10} \\y_{11} \\y_{12} \\\vdots \\y_{19} \\y_{20} \\\vdots \\y_{n\; 9}\end{bmatrix}}}} & {{Equation}\mspace{14mu}(16)}\end{matrix}$

This Equation (16) is a normal equation for calculating coefficient seeddata. By solving this normal equation by a generic solution such as asweeping-out (Gauss-Jordan elimination) method, items of the coefficientseed data w_(i0) to w_(i9) (i=1 to n) can be obtained.

FIG. 17 shows a concept of the above-described coefficient seed dataproduction method. From an HD signal as a teacher signal (secondlearning signal), a plurality of SD signals as a student signal (firstlearning signal) is produced. It is to be noted that SD signals havingdifferent resolutions are produced by changing frequency characteristicsof a thinning filter that is used when producing an SD signal from an HDsignal.

By using SD signals having different resolutions, items of coefficientseed data each having different resolution-improving effects can beproduced. For example, assuming that there are an SD signal from which amore blurred image is obtained and an SD signal from which a lessblurred image is obtained, coefficient seed data having largerresolution-improving effects is produced through learning by use of theSD signal for the more blurred image, while coefficient seed data havingsmaller resolution-improving effects is produced through learning by useof the SD signal for the less blurred image.

Further, by adding noise to each of SD signals having differentresolutions, noise-added SD signals are produced. By varying a quantityof noise to be added, SD signals having different noise quantities addedthereto are produced, thereby producing items of coefficient seed datahaving different noise rejection effects. For example, assuming thatthere are an SD signal to which more noise is added and an SD signal towhich less noise is added, coefficient seed data having larger noiserejection effect is produced through learning by use of the SD signalhaving more noise added thereto, while coefficient seed data havingsmaller noise rejection effect is produced through learning by use ofthe SD signal having less noise added thereto.

The quantity of noise to be added is adjusted by varying a value of Gif, for example as shown in Equation (17), noise n is added to a pixelvalue x of an SD signal to thereby produce a pixel value x′ of anose-added SD signal.x′=x+G·n  Equation (17)

For example, a parameter r that changes frequency characteristics isvaried in nine steps of 0 to 8 and a parameter z that changes thequantity of noise to be added is also varied in nine steps of 0 to 8 tothereby produce a total of 81 species of SD signals. Through learningbetween the thus produced plurality of SD and HD signals, thecoefficient seed data is produced. These parameters r and z correspondto the parameters r and z in the apparatus 100B of FIG. 16 forprocessing the image signal.

FIG. 18 shows a configuration of the apparatus 200B for producingcoefficient seed data that produces items of the coefficient seed dataw_(i0) to w_(i9) to be stored in the ROM 112 of FIG. 16. In this FIG.18, the components that correspond to those in FIG. 6 are indicated bythe same symbols and their detailed explanation will be omitted.

The apparatus 200B for producing the coefficient seed data comprises anSD signal production circuit 202B for obtaining an SD signal as astudent signal (first learning signal) by performing horizontal andvertical thinning processing on an HD signal as a teacher signal (secondlearning signal) input to the input terminal 201. This SD signalproduction circuit 202B is supplied with parameters r and z as a controlsignal. In accordance with the parameter r, frequency characteristics ofthe thinning filter used to produce the SD signal from the HD signal arevaried. Further, in accordance with the parameter z, the quantity ofnoise to be added to the SD signal is varied.

The apparatus 200B for producing the coefficient seed data furthercomprises a learning pair storage section 210B. Based on the class codeCa obtained by the class categorization circuit 204 and the parameters rand z supplied to the SD signal production circuit 202B, this learningpair storage section 210B stores, for each class, as learning pair data,a prediction tap and teacher data that are extracted respectively by theprediction tap extraction circuit 207 and the teacher data extractioncircuit 209 corresponding to each target position in the SD signal withthem being correlated with values of the parameters r and z.

The apparatus 200B for producing the coefficient seed data furthercomprises a computation circuit 211B. This computation circuit 211Bproduces a normal equation (see Equation (16)) for calculating items ofthe coefficient seed data w_(i0) to w_(i9) for each class using multipleitems of learning pair data stored in the learning pair storage section210B. It is to be noted that in this case, the computation circuit 211Bproduces a normal equation for each output pixel position (each of thepositions of y₁ to y₄). That is, the computation circuit 211B produces anormal equation for each combination of classes and output pixelpositions. Further, this computation circuit 211B calculates items ofthe coefficient seed data w_(i0) to w_(i9) for each combination ofclasses and output pixel positions by solving each normal equation.

The other components of the apparatus 200B for producing the coefficientseed data are configured the same way as those of the apparatus 200 forproducing the coefficient data shown in FIG. 6.

The following will describe operations of the apparatus 200B forproducing the coefficient seed data shown in FIG. 18.

Horizontal and vertical thinning processing is performed by the SDsignal production circuit 202B on an HD signal input to the inputterminal 201, to produce an SD signal as a student signal. In this case,the SD signal production circuit 202B is supplied with the parameters rand z as the control signal, to serially produce a plurality of SDsignals having step-wise changed frequency characteristics andadded-noise quantities. The SD signals produced by this SD signalproduction circuit 202B are supplied to the class tap extraction circuit203 and also, via the time-adjusting delay circuit 206, to theprediction tap extraction circuit 207.

Operations of the class tap extraction circuit 203, the classcategorization circuit 204, the DR processing circuit 205, the teacherdata extraction circuit 209, and the prediction tap extraction circuit207 are the same as those in the apparatus 200 for producing thecoefficient data shown in FIG. 6 and so their explanation will beomitted.

The learning pair storage section 210B is supplied with a class code Caobtained by the class categorization circuit 204, a prediction tapextracted by the prediction tap extraction circuit 207, teacher dataextracted by the teacher data extraction circuit 209, and the sameparameters r and z as those supplied to the SD signal production circuit202B.

Then, in this learning pair storage section 210B, a prediction tap andteacher data are stored as learning pair data. In this case, based onthe class code Ca and the parameters r and z, each of the items oflearning pair data is stored with them being correlated with values ofthe parameters r and z.

The computation circuit 211B generates a normal equation (see Equation(16)) for calculating items of the coefficient seed data w_(i0) tow_(i9) for each combination of classes and output pixel positions usingmultiple items of learning pair data stored in the learning pair storagesection 210B. Furthermore, this computation circuit 211B solves eachnormal equation to calculate items of the coefficient seed data w_(i0)to w_(i9) for each combination of the classes and the output pixelpositions. Items of the coefficient seed data w_(i0) to w_(i9) thusobtained by the computation circuit 211B are stored in the coefficientmemory 212.

Although a processing apparatus for it is not shown, the processing inthe apparatus 200B for producing the coefficient seed data shown in FIG.18 can also be performed by software. A processing procedure in thiscase is roughly the same as that for coefficient data productionprocessing shown in FIG. 12.

However, corresponding to an HD signal as a teacher signal input at stepST32, SD signals as student signals which have frequency characteristicsand noise-added quantities corresponding to all combinations of valuesof the parameter r and z are produced and each used to perform learningprocessing at step ST34.

At step ST40, the process performs supplementation to obtain a normalequation given in Equation (16) for each class by using the class codeCa acquired at step ST38, pixel data xi of the prediction tap extractedat step ST39, items of pixel data (items of teacher data) y₁ to y₄ ofthe HD signal input at step ST32 corresponding to a target position inthe SD signal, and the parameters r and z (see Equations (14) and (15)).

Further, at step ST42, instead of calculating the coefficient data Wi,the process obtains items of coefficient seed data w_(i0) to w_(i9) ofeach class by solving the normal equation produced by theabove-described supplementation processing at step ST40 and, at stepST43, saves these items of coefficient seed data w_(i0) to w_(i9) in thecoefficient memory and ends at step ST44.

FIG. 19 shows a configuration of a LUT production apparatus 300B forproducing a correspondence relationship between a class code Ca and aclass code Cb in the lookup table 105 shown in FIG. 16. In this FIG. 19,the components that correspond to those in FIG. 7 are indicated by thesame symbols and their detailed explanation will be omitted.

The LUT production apparatus 300B comprises an SD signal productioncircuit 302B for obtaining an SD signal as a student signal (firstlearning signal) by performing horizontal and vertical thinningprocessing on an HD signal as a teacher signal (second learning signal)input to the input terminal 301. This SD signal production circuit 302Bis supplied with the parameters r and z as the control signal. Inaccordance with the parameter r, frequency characteristics of thethinning filter used to produce the SD signal from the HD signal arevaried. In accordance with the parameter z, the quantity of noise to beadded to the SD signal is varied.

The LUT production apparatus 300B further comprises an ROM 319 forstoring items of coefficient seed data w_(i0) to w_(i9) of each class.This coefficient seed data is coefficient data of a production equationfor producing items of coefficient data Wi (i=1 to n) to be stored inthe coefficient memory 306. This ROM 319 corresponds to the ROM 112 inthe apparatus 100B for processing the image signal of FIG. 16 and storesthe same coefficient seed data as that stored in this ROM 112.

The LUT production apparatus 300B further comprises a coefficientproduction circuit 320 for producing coefficient data Wi, which is usedin an estimate equation and corresponds to values of the parameters rand z for each combination of classes and output pixel positions basedon the above-described Equation (9) by using the coefficient seed dataof each class and the values of the parameters r and z. Into thiscoefficient production circuit 320, items of coefficient seed dataw_(i0) to w_(i9) are loaded from the ROM 319. Further, this coefficientproduction circuit 113 is supplied with the parameters r and z providedto the SD signal production circuit 302B.

The production of coefficient data Wi in this coefficient productioncircuit 320 is performed each time the values of the parameters r and zare changed. The coefficient data Wi of each class produced in thiscoefficient production circuit 320 is stored in the above-describedcoefficient memory 306.

The components of the LUT production apparatus 300B are configured thesame way as those of the LUT production apparatus 300 shown in FIG. 7.

The following will describe operations of the LUT production apparatus300B shown in FIG. 19.

Horizontal and vertical thinning processing is performed by the SDsignal production circuit 302B on an HD signal input to the inputterminal 301, to produce an SD signal as a student signal. In this case,the SD signal production circuit 302B is supplied with the parameters rand z as the control signal, to serially produce a plurality of SDsignals having step-wise changed frequency characteristics andadded-noise quantities. The SD signals produced by this SD signalproduction circuit 302B are supplied to the class tap extraction circuit303 and also, via the time-adjusting delay circuit 307, to theprediction tap extraction circuit 308.

The parameters r and z same as those supplied to the SD signalproduction circuit 302B are supplied also to the coefficient productioncircuit 320. In this coefficient production circuit 320, each time thevalues of the parameters r and z are changed, coefficient data Wi ofeach class corresponding to these values of the parameters r and z isproduced. Then, this coefficient data Wi is stored in the coefficientmemory 306.

The other circuits operate the same way as those in the LUT productionapparatus 300 shown in FIG. 7. Therefore, in the error memory 316, anaccumulated value is stored of each output class at each of the inputclasses obtained on the basis of the HD signal input to the inputterminal 301 and each of the SD signals produced by the SD signalproduction circuit 302B.

Then, in the error minimum class detection circuit 317, based on anaccumulated value of each output class at each of the input classes,which is stored in the error memory 316, an output class in which theaccumulated value is minimizes is allocated to each of the inputclasses, to acquire a correspondence relationship (see FIG. 4) betweenthe class codes Ca and Cb, and this correspondence relationship isstored in the memory 318.

Further, although a processing apparatus for it is not shown, theprocessing in the LUT production apparatus 300B of FIG. 19 can also berealized by software. A processing procedure in this case is roughly thesame as that for the LUT production processing shown in FIG. 13.

However, corresponding to the HD signal as a teacher signal input atstep ST52, SD signals as student signals which have frequencycharacteristics and noise-added quantities corresponding to allcombinations of values of the parameter r and z are produced and eachused to perform processing to obtain an accumulated value of each outputclass at each of the input classes at step ST54.

Further, in that processing, at step ST60, items of coefficient seeddata w_(i0) to w_(i9) that correspond to the class code Ca acquired atstep ST58 and values of the parameters r and z that correspond to the SDsignal at step ST54 are used to produce coefficient data Wi-a based on aproduction equation of the above-described Equation (9), and thiscoefficient data Wi-a is then used to calculate an error E (p)=y₀−y.

Similarly, at step ST62, items of coefficient seed data w_(i0) to w_(i9)that correspond to the class number q and values of the parameters r andz that correspond to the SD signal produced at step ST54 are used toproduce coefficient data Wi-q based on a production equation of theabove-described Equation (9), and this coefficient data Wi-q is thenused to calculate an error E (q)=y₀−y_(q).

At step ST67, based on an accumulated value of each output class at eachof the input classes obtained by using each SD signal, an output classin which the accumulated value is minimized is allocated to each of theinput classes, to acquire a correspondence relationship (see FIG. 4)between the class codes Ca and Cb.

Although detailed description will be omitted, such a configuration maybe thought of that in the above-described apparatus 100A for processingthe image signal shown in FIG. 14, items of coefficient data Wis and Wicto be stored in its coefficient memory 106A are produced on the basis ofa production equation of Equation (9) from items of the coefficient seeddata w_(i0) to w_(i9) and values of the parameters r and z likecoefficient data Wi to be stored in the coefficient memory 106 in theapparatus 100B for processing the image signal shown in FIG. 16.

Further, although in the above-described embodiments, the parameter rthat determines resolution and the parameter z that determines a degreeof noise rejection have been included in the production equation ofEquation (9), the kinds and the number of the parameters are not limitedto them.

Further, in the above-described apparatus 100 for processing the imagesignal shown in FIG. 1 and the above-described apparatus 100B forprocessing the image signal shown in FIG. 16, if the area information ARis “1”, that is, the dynamic range DR is less than a threshold value Th,addition mean value (y_(1-a)+y_(1-b))/2 through (y_(4-a)+y_(4-b))/2 ofitems of the pixel data y_(1-a) to y_(4-a) calculated by using thecoefficient data Wi-a obtained corresponding to the class code Caobtained on the basis of the class tap and items of the pixel datay_(1-b) to y_(4-b) calculated by using the coefficient data Wi-bobtained corresponding to the class code Cb obtained by converting thisclass code Ca are output as items of pixel data y₁ to y₄ that constitutethe HD signal.

However, the number of classes to be subject to addition mean is notlimited to two; such the addition mean may be considered to be performedon three or more classes and a result one is output. For example, whenthe addition mean is performed on three classes, a class code thatindicates the third class can be obtained by obtaining an addition meanvalue (y_(a)+y_(b))/2 of the two classes at the predictive computationcircuit 309 and obtaining the error E (p) by subtracting the additionmean value (y_(a)+y_(b))/2 from the teacher data y₀ at the predictionerror calculation circuit 313. It is thus possible to allocate an outputclass corresponding to each of the input classes (two classes) at theerror minimum class detection circuit 317, thereby obtaining the thirdclass code that corresponds to this output class.

Further, in the above-described apparatus 100 for processing the imagesignal shown in FIG. 1 and the above-described apparatus 100B forprocessing the image signal shown FIG. 16, if DR≧Th, items of pixel datay_(1-a) to y_(4-a) calculated by using coefficient data Wi-acorresponding to a class code Ca have been output as items of pixel datay₁ to y₄ that constitute an HD signal and, if DR<Th, an addition meanvalue (y_(1-a)+y_(1-b))/2 through (y_(4-a)+y_(4-b))/2 of items of pixeldata y_(1-a) to y_(4-a) and y_(1-b) to y_(4-b) calculated by using itemsof coefficient data Wi-a and Wi-b corresponding to the class codes Caand Cb, respectively, have been output as items of pixel data y₁ to y₄that constitute the HD signal.

However, the operations in a case where DR≧Th and those in a case whereDR<Th may be considered to be made opposite. In this case, coefficientdata Wi to be stored in the coefficient memory 106 is supposed to bebased on a result of learning between a student signal that correspondsto the SD signal and a teacher signal that corresponds to the HD signalby use of such a portion of the dynamic range DR as to be smaller thanthe threshold value Th. It is to be noted that the operation in thecases of DR≧Th and DR<Th can be made opposite similarly also in theapparatus 100A for processing the image signal shown in FIG. 14.

Further, in the above-described embodiments, a possible area of thedynamic range has been divided into two by using a threshold value Thbeforehand; and if DR≧Th, “0” may be output as the area information, andif DR<Th, “1” may be output as the area information AR.

However, it may be considered that by dividing the possible area of thedynamic range DR into three or more to acquire area information AR thatindicates which one of these sub-divided areas the dynamic range DRbelongs to so that processing may be performed in accordance with thisarea information AR. In this case, for example, if the dynamic range DRbelongs to one sub-divided area, the same processing as theabove-described processing in the case of DR≧Th is to be performed,whereas if the dynamic range DR belongs to another sub-divided areadifferent from that one sub-divided area, the same processing as theabove-described processing in the case of DR<Th is to be performed.

Further, although the above embodiments have been described withreference to an example of converting an SD signal into an HD signalhaving twice the number of pixels horizontally and vertically, adirection in which the number of pixels is increased is not limited tothe horizontal and vertical directions and may be considered to be atime direction (frame direction). Also to the case of, oppositely,obtaining an SD signal having a decreased number of pixels from an HDsignal, the present invention can be applied similarly. That is, thepresent invention can be applied generally to the case of converting afirst image signal into a second image signal that has the same numberof pixels as or a different number of pixels from that of this firstimage signal.

Further, although the above embodiments have been described withreference to an example where an information signal comprised ofmultiple items of information data is an image signal comprised ofmultiple items of pixel data, the present invention can be similarlyapplied to a case where the information signal is any other signal, forexample, an audio signal. In the case of an audio signal, it iscomprised of multiple items of sample data.

By the present invention, when converting a first information signalcomprised of multiple items of information data into a secondinformation signal comprised of multiple items of information data,multiple items of information data located in a periphery of a targetposition in the first information signal is extracted as a class tapand, if a dynamic range obtained from information data in this class tapbelongs to one area, information data that constitutes the secondinformation signal is obtained by using coefficient data thatcorresponds to a first class code obtained by class-categorizing thatclass tap based on a result of learning between a student signal (firstlearning signal) and a teacher signal (second learning signal) by use ofsuch a portion of the dynamic range as to belong to that one area, andif the dynamic range belongs another area different from that one area,information data that constitutes the second information signal isobtained by performing addition mean on information data calculated byusing coefficient data that corresponds to the first class code andinformation data calculated by using coefficient data that correspondsto a second class code obtained by converting this first class code, sothat it is possible to well obtain information data that constitutes thesecond information signal no matter whether the dynamic range is largeor small.

Further, by the present invention, when converting a first informationsignal comprised of multiple items of information data into a secondinformation signal comprised of multiple items of information data,multiple items of information data located in a periphery of a targetposition in the first information signal is extracted as a class tapand, if a dynamic range obtained from information data in this class tapbelongs to one area, information data that constitutes the secondinformation signal is obtained by using coefficient data thatcorresponds to a first class code obtained by class-categorizing theclass tap based on a result of learning between a student signal (firstlearning signal) and a teacher signal (second learning signal) by use ofsuch a portion of the dynamic range as to belong to that one area, andif the dynamic range belongs to another area other than that one area,information data that constitutes the second information signal isobtained by using coefficient data based on a result of learning betweenthe student signal and the teacher signal without class categorization,so that it is possible to well obtain information data that constitutesthe second information signal no matter whether the dynamic range islarge or small.

INDUSTRIAL APPLICABILITY

When converting a first information signal into a second informationsignal, an apparatus for processing an information signal etc. relatedto the present invention can well obtain information data thatconstitutes the second information signal no matter whether a dynamicrange is large or small and so can be applied, for example, to the caseof converting a standard TV signal (SD signal) corresponding to astandard or low resolution into a high-resolution signal (HD signal).

1. An apparatus for converting a first information signal of standarddefinition comprised of multiple items of information data into a secondinformation signal of high definition image data comprised of multipleitems of information data, the apparatus comprising: class tapextraction means for extracting as a class tap multiple items ofinformation data located in a periphery of a target position in thefirst information signal based on the first information signal; classcategorization means for obtaining a first class code by categorizingthe class tap extracted by the class tap extraction means as any one ofa plurality of classes based on the class tap; dynamic range processingmeans for detecting a dynamic range which is a difference between amaximum value and a minimum value of the multiple items of informationdata contained in the class tap extracted by the class tap extractionmeans based on the class tap, to obtain area information that indicateswhich one of a plurality of sub-divided areas obtained by dividing apossible area of the dynamic range into plural ones the dynamic rangebelongs to; class code conversion means for converting the first classcode obtained by the class categorization means into one or a pluralityof second class codes each corresponding to the first class code;prediction tap extraction means for extracting as a prediction tapmultiple items of information data located in a periphery of the targetposition in the first information signal based on the first informationsignal; first coefficient data generation means for generating firstcoefficient data, which is used in an estimate equation corresponding tothe first class code obtained by the class categorization means; secondcoefficient data generation means for generating second coefficientdata, which is used in the estimate equation, corresponding to one orthe plurality of second class codes, respectively, obtained throughconversion by the class code conversion means; first computation meansfor calculating information data based on the estimate equation, byusing the first coefficient data generated by the first coefficient datageneration means and the prediction tap extracted by the prediction tapextraction means; second computation means for calculating informationdata based on the estimate equation, by using the second coefficientdata generated by the second coefficient data generation means and theprediction tap extracted by the prediction tap extraction means; andaddition means for outputting the information data calculated by thefirst computation means as information data that constitutes the secondinformation signal corresponding to a target position in the firstinformation signal if the dynamic range belongs to one sub-divided areaaccording to the area information obtained by the dynamic rangeprocessing means and, if the dynamic range belongs to anothersub-divided area different from the one sub-divided area, outputtingdata obtained by performing addition mean on the information datacalculated by the first computation means and that calculated by thesecond computation means as the information data that constitutes thesecond information signal corresponding to the target position in thefirst information signal, wherein the first coefficient data generatedby the first coefficient data generation means and the secondcoefficient data generated by the second coefficient data generationmeans are based on a result of learning between a first learning signalthat corresponds to the first information signal and a second learningsignal that corresponds to the second information signal by use of sucha portion of the dynamic range as to belong to the one sub-divided area;and wherein the class code conversion means converts the first classcode into the second class code in such a manner that the addition meanvalue of the information data calculated by the first computation meanscorresponding to the first class code and the information datacalculated by the second computation means corresponding to the secondclass code may most approach a true value of the information data thatconstitutes the second information signal.
 2. The apparatus forprocessing the information signal according to claim 1, wherein thedynamic range processing means obtains area information that indicateswhether the dynamic range is less than a threshold value or not lessthan the threshold value.
 3. The apparatus for processing theinformation signal according to claim 2, wherein if the dynamic range isnot less than the threshold value, the addition means outputsinformation data obtained by the first computation means as informationdata that constitutes the second information signal corresponding to atarget position in the first information signal and, if the dynamicrange is less than the threshold value, outputs data obtained byperforming addition mean on information data obtained by the firstcomputation means and that obtained by the second computation means asinformation data that constitutes the second information signalcorresponding to the target position in the first information signal. 4.The apparatus for processing the information signal according to claim1, wherein the first coefficient data generation means and the secondcoefficient data generation means each comprise: storage means forstoring coefficient data which is obtained beforehand and which is usedin the estimate equation of each class; and coefficient data readingmeans for reading coefficient data that corresponds to a class indicatedby a class code from the storage means.
 5. The apparatus for processingthe information signal according to claim 1, wherein the firstcoefficient data generation means and the second coefficient datageneration means each comprise: storage means for storing coefficientseed data that is obtained beforehand for each class and is coefficientdata in a production equation, which includes a predetermined parameter,for producing coefficient data to be used in the estimate equation; andcoefficient data production means for producing coefficient data to beused in the estimate equation based on the production equation by usingthe coefficient seed data corresponding to a class indicated by a classcode stored in the storage means.
 6. The apparatus for processing theinformation signal according to claim 1, wherein the class codeconversion means is configured by a lookup table in which acorrespondence relationship between the first class code and the secondclass code is stored.
 7. The apparatus for processing the informationsignal according to class 1, wherein the information signal is an imagesignal or an audio signal.
 8. An image quality improvement methodexecuted by a processor for converting a first information signal ofstandard definition comprised of multiple items of information data intoa second information signal of high definition image data comprised ofmultiple items of information data, the method comprising: a class tapextraction step of extracting as a class tap multiple items ofinformation data located in a periphery of a target position in thefirst information signal based on the first information signal; a classcategorization step of obtaining a first class code by categorizing theclass tap extracted by the class tap extraction step as any one of aplurality of classes based on the class tap; a dynamic range processingstep of detecting a dynamic range which is a difference between amaximum value and a minimum value of the multiple items of informationdata contained in the class tap extracted by the class tap extractionstep based on the class tap, to obtain area information that indicateswhich one of a plurality of sub-divided areas obtained by dividing apossible area of the dynamic range into plural ones the dynamic rangebelongs to; a class code conversion step of converting a first classcode obtained by the class categorization step into one or a pluralityof second class codes each corresponding to the first class code; aprediction tap extraction step of extracting as a prediction tapmultiple items of information data located in a periphery of the targetposition in the first information signal based on the first informationsignal; a first coefficient data generation step of generating firstcoefficient data, which is used in an estimate equation corresponding tothe first class code obtained by the class categorization step; a secondcoefficient data generation step of generating second coefficient data,which is used in the estimate equation, corresponding to one or theplurality of second class codes, respectively, obtained throughconversion by the class code conversion step; a first computation stepof calculating information data based on the estimate equation, by usingthe first coefficient data generated by the first coefficient datageneration step and the prediction tap extracted by the prediction tapextraction step; a second computation step of calculating informationdata based on the estimate equation, by using the second coefficientdata generated by the second coefficient data generation step and theprediction tap extracted by the prediction tap extraction step; and anaddition step of outputting the information data calculated by the firstcomputation step as information data that constitutes the secondinformation signal corresponding to a target position in the firstinformation signal if the dynamic range belongs to one sub-divided areaaccording to the area information obtained by the dynamic rangeprocessing step and, if the dynamic range belongs to another sub-dividedarea different from the one sub-divided area, outputting data obtainedby performing addition mean on the information data calculated by thefirst computation step and that calculated by the second computationstep as the information data that constitutes the second informationsignal corresponding to the target position in the first informationsignal, wherein the first coefficient data generated by the firstcoefficient data generation step and the second coefficient datagenerated by the second coefficient data generation step are based on aresult of learning between a first learning signal that corresponds tothe first information signal and a second learning signal thatcorresponds to the second information signal by use of such a portion ofthe dynamic range as to belong to the one sub-divided area; and whereinin the class code conversion step, the first class code is convertedinto the second class code in such a manner that the addition mean valueof the information data calculated by the first computation stepcorresponding to the first class code and the information datacalculated by the second computation step corresponding to the secondclass code may most approach a true value of the information data thatconstitutes the second information signal.
 9. A non-transitory recordingmedium on which a computer-readable program is recorded, the programcausing a computer to perform an image quality improvement method,executed by a processor, for converting a first information signal ofstandard definition comprised of multiple items of information data intoa second information signal of high definition image data comprised ofmultiple items of information data, the method comprising: a class tapextraction step of extracting as a class tap multiple items ofinformation data located in a periphery of a target position in thefirst information signal based on the first information signal; a classcategorization step of obtaining a first class code by categorizing theclass tap extracted by the class tap extraction step as any one of aplurality of classes based on the class tap; a dynamic range processingstep of detecting a dynamic range which is a difference between amaximum value and a minimum value of the multiple items of informationdata contained in the class tap extracted by the class tap extractionstep based on the class tap, to obtain area information that indicateswhich one of a plurality of sub-divided areas obtained by dividing apossible area of the dynamic range into plural ones the dynamic rangebelongs to; a class code conversion step of converting a first classcode obtained by the class categorization step into one or a pluralityof second class codes each corresponding to the first class code; aprediction tap extraction step of extracting as a prediction tapmultiple items of information data located in a periphery of the targetposition in the first information signal based on the first informationsignal; a first coefficient data generation step of generating firstcoefficient data, which is used in an estimate equation corresponding tothe first class code obtained by the class categorization step; a secondcoefficient data generation step of generating second coefficient data,which is used in the estimate equation, corresponding to one or theplurality of second class codes, respectively, obtained throughconversion by the class code conversion step; a first computation stepof calculating information data based on the estimate equation, by usingthe first coefficient data generated by the first coefficient datageneration step and the prediction tap extracted by the prediction tapextraction step; a second computation step of calculating informationdata based on the estimate equation, by using the second coefficientdata generated by the second coefficient data generation step and theprediction tap extracted by the prediction tap extraction step; and anaddition step of outputting the information data calculated by the firstcomputation step as information data that constitutes the secondinformation signal corresponding to a target position in the firstinformation signal if the dynamic range belongs to one sub-divided areaaccording to the area information obtained by the dynamic rangeprocessing step and, if the dynamic range belongs to another sub-dividedarea different from the one sub-divided area, outputting data obtainedby performing addition mean on the information data calculated by thefirst computation step and that calculated by the second computationstep as the information data that constitutes the second informationsignal corresponding to the target position in the first informationsignal, wherein the first coefficient data generated by the firstcoefficient data generation step and the second coefficient datagenerated by the second coefficient data generation step are based on aresult of learning between a first learning signal that corresponds tothe first information signal and a second learning signal thatcorresponds to the second information signal by use of such a portion ofthe dynamic range as to belong to the one sub-divided area; and whereinin the class code conversion step, the first class code is convertedinto the second class code in such a manner that the addition mean valueof the information data calculated by the first computation stepcorresponding to the first class code and the information datacalculated by the second computation step corresponding to the secondclass code may most approach a true value of the information data thatconstitutes the second information signal.